Decoding method, encoding method, decoding device, and encoding device

Abstract

The decoding method decodes a plurality of three-dimensional points, acquires a plurality of nodes included in a first slice and having an octree structure from a bitstream (S301), acquires information for deriving a shape of a first node among the plurality of nodes from the bitstream (S302), and decodes the first node in accordance with the information (S303). The shape of the first node is different from a predetermined shape of another node among the plurality of nodes. For example, the shape may also be a cuboid shape, rather than a cubic shape.

Description

Technical Field

[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device. Background Art

[0002] Devices and services that effectively utilize three-dimensional data are expected to become more common in a wide range of fields, including computer vision for autonomous operation of cars or robots, map information, surveillance, infrastructure inspection, and image distribution. Three-dimensional data is obtained through various methods, such as distance sensors such as rangefinders, stereo cameras, or a combination of multiple monocular cameras.

[0003] As one of the methods of expressing three-dimensional data, there is a method called point cloud that expresses the shape of a three-dimensional structure through a group of points in a three-dimensional space. In a point cloud, the position and color of a point group are stored. Although point clouds are expected to become the mainstream method of expressing three-dimensional data, the amount of data in a point group is very large. Therefore, in the accumulation or transmission of three-dimensional data, like two-dimensional dynamic images (for example, MPEG-4AVC or HEVC standardized by MPEG), it is necessary to compress the data volume through encoding.

[0004] In addition, regarding compression of point clouds, some support is provided by a public library (Point Cloud Library) or the like that performs processing related to point clouds.

[0005] In addition, there is known a technique for searching and displaying facilities located around a vehicle using three-dimensional map data (see, for example, Patent Document 1).

[0006] Prior Art Literature

[0007] Patent Literature

[0008] Patent Document 1: International Publication No. 2014 / 020663 Summary of the invention

[0009] Problems to be solved by the invention

[0010] In such encoding methods and decoding methods, it is desired to improve the encoding efficiency.

[0011] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve encoding efficiency.

[0012] Means for solving problems

[0013] A decoding method involved in one embodiment of the present invention decodes multiple three-dimensional points, wherein multiple nodes having an octree structure and contained in a first slice are obtained from a bit stream, information for deriving a shape of a first node among the multiple nodes is obtained from the bit stream, and the first node is decoded according to the information, the shape being different from specified shapes of other nodes among the multiple nodes.

[0014] A coding method involved in one embodiment of the present invention encodes multiple three-dimensional points, wherein multiple nodes having an octree structure and contained in a first slice are encoded to generate a bit stream, and information for deriving the shape of a first node among the multiple nodes is stored in the bit stream, wherein the shape is different from the prescribed shapes of other nodes among the multiple nodes.

[0015] Effects of the Invention

[0016] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a diagram showing an example of a primitive point group according to an embodiment of the present invention.

[0018] Figure 2 This is a diagram showing an example of a pruned octree according to an embodiment of the present invention.

[0019] Figure 3 This is a diagram showing an example of two-dimensionally displaying leaf nodes according to the embodiment.

[0020] Figure 4 This is a diagram for explaining a method for generating a centroid vertex according to an embodiment.

[0021] Figure 5 This is a diagram for explaining a method for generating a centroid vertex according to an embodiment.

[0022] Figure 6 This is a diagram showing an example of vertex information according to an embodiment.

[0023] Figure 7 It is a diagram showing an example of a triangle aggregate surface involved in the embodiment.

[0024] Figure 8 It is a diagram for explaining the restoration process of the point group involved in the embodiment.

[0025] Fig. 9 It is a diagram showing an example of slice division according to the embodiment.

[0026] Fig.10This is a diagram showing examples of vertices according to an embodiment.

[0027] Fig.11 This is a diagram showing an example of a triangle aggregate surface to be generated according to the embodiment.

[0028] Fig.12 This is a diagram showing an example of a triangle aggregate surface in which edge vertices are not generated according to an embodiment.

[0029] Fig.13 It is a diagram showing an example of a restored point group according to an embodiment.

[0030] Fig.14 This is a diagram showing examples of vertices according to an embodiment.

[0031] Fig.15 It is a diagram showing an example of a triangle aggregate surface involved in the embodiment.

[0032] Fig.16 It is a diagram showing an example of transmission information involved in the embodiment.

[0033] Fig.17 This is a diagram showing a syntax example of a GDU header according to an embodiment of the present invention.

[0034] Fig.18 It is a diagram showing a setting example of the adjustment width of the unspecified width node according to the embodiment.

[0035] Fig.19 This is a flowchart of the encoding process performed by the encoding device involved in the embodiment.

[0036] Fig. 20 This is a flowchart of a decoding process performed by the decoding device involved in the embodiment.

[0037] Fig.21 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0038] Fig. 22 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0039] Fig.23 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0040] Fig.24 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0041] Fig.25 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0042] Fig.26 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0043] Fig. 27 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0044] Fig.28 It is a diagram showing the relationship between the node width involved in the implementation mode and the width of the connotation point coordinates.

[0045] Fig.29 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0046] Fig.30 It is a diagram showing a setting example of the unspecified width node involved in the embodiment.

[0047] Fig.31 This is a diagram showing a syntax example of the GPS and GDU header involved in the implementation method.

[0048] Fig.32 This is a diagram showing examples of slices and nodes involved in an implementation manner.

[0049] Fig.33 This is a diagram showing examples of slices and nodes involved in an implementation manner.

[0050] Fig.34 This is a diagram showing an example of processing when omission is performed according to the embodiment.

[0051] Fig.35 This is a diagram showing a syntax example of the GPS and GDU header involved in the implementation method.

[0052] Fig.36 This is a flowchart of the process of determining whether to omit the process of adjusting the start end according to the embodiment.

[0053] Fig.37 This is a flowchart of a process for determining whether to omit the adjustment process at the terminal according to the embodiment.

[0054] Fig.38 This is a flowchart of the node position determination process according to the embodiment.

[0055] Fig.39 This is a flowchart of the node width determination process according to the embodiment.

[0056] Fig.40 This is a flowchart of the decoding process involved in the implementation mode.

[0057] Fig.41This is a block diagram of a decoding device according to an embodiment.

[0058] Fig.42 This is a flowchart of the encoding process involved in the implementation mode.

[0059] Fig.43 This is a block diagram of a decoding device according to an embodiment. DETAILED DESCRIPTION

[0060] A three-dimensional data decoding method according to one embodiment of the present disclosure decodes a plurality of three-dimensional points, wherein a plurality of nodes having an octree structure and contained in a first slice are obtained from a bit stream, information for deriving the shape of a first node among the plurality of nodes is obtained from the bit stream, and the first node is decoded according to the information, wherein the shape is different from the prescribed shapes of other nodes among the plurality of nodes. Thus, a node having a shape different from the prescribed shape can be set. Thus, a variable node can be set according to the size of the slice or the distribution of the point group. Thus, there is a possibility of improving the coding efficiency.

[0061] For example, the shape may be a rectangular parallelepiped shape instead of a cube shape. For example, the end of the first slice may be consistent with the end of one of the plurality of nodes. Thus, in the case where the node end is inconsistent with the slice end, the node end can be made consistent with the slice end. Thus, the situation where no vertex is generated at the node end can be suppressed, and thus the generation of a blank area at the slice boundary can be suppressed. Thus, the accuracy of the decoded point group can be improved.

[0062] For example, the information may indicate the size of the shape or the positions of both ends of the side of the first node. Thus, the decoding device can generate a node having a shape different from a predetermined shape using the information.

[0063] For example, the information may include adjustment information for adjusting the prescribed shape to the prescribed shape. Thus, the decoding device can generate a node having a shape different from the prescribed shape using the adjustment information. In addition, there is a possibility that the amount of information can be reduced compared to the case where the absolute amount of information of the transmission position is transmitted.

[0064] For example, the decoding may be performed according to a compression method that approximates the plurality of three-dimensional points with a plane or a curved surface within the first node. For example, the compression method may be a Triangle-Soup compression method.

[0065] For example, the shape may be determined in order to generate the plane or the curved surface in the first node. Thus, by setting a node having a shape different from a predetermined shape, a plane or a curved surface can be generated in the first node.

[0066] For example, the side of the shape may have a vertex, and the plane or the curved surface may intersect the side at the vertex. Thus, by setting a node of a shape different from the predetermined shape, a plane or a curved surface can be generated in the first node.

[0067] For example, the first node may be arranged in contact with the second slice adjacent to the first slice. Thus, for example, when the node end is inconsistent with the slice end, the situation where no vertex is generated at the node end can be suppressed, and thus the generation of a blank area at the slice boundary can be suppressed. For example, when only nodes of a specified shape are set at the slice boundary, there is a situation where the nodes are divided by the slice boundary. In this case, the point group in the second slice adjacent to the first slice cannot be used to encode or decode the first node, so there is a possibility that the restoration accuracy of the three-dimensional point group near the slice boundary will deteriorate. Therefore, in this method, by setting a node of a shape different from the specified shape, for example, the end of the first node can be made consistent with the end of the first slice. Thus, the restoration accuracy of the three-dimensional point group near the slice boundary can be suppressed. In addition, if this method is applied to the triangle set method, the situation where the edge vertices are not properly generated can be suppressed.

[0068] For example, the information may be set for each slice, and the information of the second slice may be used to derive the shape of a second node among a plurality of nodes having an octree structure and included in the second slice, and the shape of the second node may be different from the prescribed shape. Thus, the shape of the node may be set for each slice.

[0069] For example, the size of the predetermined shape may be expressed by a power of 2, and the size of the shape may be different from the size expressed by a power of 2.

[0070] For example, the shape of the first node may be defined by a first length along a first direction, a second length along a second direction, and a third length along a third direction, the first direction, the second direction, and the third direction are orthogonal to each other, and only the first length among the first length, the second length, and the third length is different from the length of the prescribed shape of nodes other than the first node, or, only the first length and the second length among the first length, the second length, and the third length are each different from the length of the prescribed shape.

[0071] For example, the first node may be closest to the origin of the first slice among the plurality of nodes in one direction among the first direction, the second direction, and the third direction, and the first direction, the second direction, and the third direction are orthogonal to each other. Thus, when the starting position of the slice does not coincide with the origin, the starting position of the node may be adjusted in accordance with the starting position of the slice, for example.

[0072] For example, the plurality of nodes may include a third node having a shape different from the prescribed shape, and the third node is farthest from the origin in the one direction among the plurality of nodes. Thus, when the end position of the slice is inconsistent with the end position of the node, the end position of the node can be adjusted in accordance with the end position of the slice, for example.

[0073] For example, when the starting position of the first slice is inconsistent with the origin, the bit stream includes the information, and when the starting position of the first slice is consistent with the origin, the bit stream does not include the information. Thus, the occurrence of processing for adjusting the shape of the node can be reduced. In addition, the transmission of information for deriving the shape of the node can be omitted. Thus, the amount of processing and the amount of data in the bit stream can be reduced.

[0074] For example, when the end position of the first slice is inconsistent with the end end of the first node, the bit stream includes the information, and when the end position of the first slice is consistent with the end end of the first node, the bit stream does not include the information. Thus, the occurrence of processing for adjusting the shape of the node can be reduced. In addition, the transmission of information for deriving the shape of the node can be omitted. Thus, the amount of processing can be reduced and the amount of data in the bit stream can be reduced.

[0075] A three-dimensional data encoding method according to one embodiment of the present disclosure encodes a plurality of three-dimensional points, wherein a plurality of nodes having an octree structure and contained in a first slice are encoded to generate a bit stream, and information for deriving the shape of a first node among the plurality of nodes is stored in the bit stream, and the shape is different from the prescribed shapes of other nodes among the plurality of nodes. Thus, a node having a shape different from the prescribed shape can be set. Thus, a variable node can be set corresponding to the size of the slice or the distribution of the point group. Thus, there is a possibility of improving the coding efficiency.

[0076] In addition, a decoding device involved in one embodiment of the present invention decodes multiple three-dimensional points and is equipped with a processor and a memory. The processor uses the memory to obtain, from a bit stream, multiple nodes having an octree structure and contained in a first slice, and obtains, from the bit stream, information for deriving a shape of a first node among the multiple nodes, and decodes the first node according to the information, wherein the shape is different from specified shapes of other nodes among the multiple nodes.

[0077] In addition, a coding device involved in one embodiment of the present invention encodes multiple three-dimensional points and is equipped with a processor and a memory. The processor uses the memory to encode multiple nodes having an octree structure and contained in a first slice, thereby generating a bit stream, and stores information for deriving the shape of a first node among the multiple nodes in the bit stream, wherein the shape is different from the prescribed shapes of other nodes among the multiple nodes.

[0078] In addition, these general or specific methods can also be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, and can also be implemented by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0079] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In addition, the embodiments described below all represent a specific example of the present disclosure. The numerical values, shapes, materials, constituent elements, configuration positions and connection methods of constituent elements, steps, and the order of steps shown in the following embodiments are examples and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims are described as arbitrary constituent elements.

[0080] (Implementation Method)

[0081] The following describes the encoding device (three-dimensional data encoding device) and decoding device (three-dimensional data decoding device) involved in this embodiment. The encoding device encodes three-dimensional data to generate a bit stream. The decoding device decodes the bit stream to generate three-dimensional data.

[0082] Three-dimensional data is, for example, three-dimensional point group data (also referred to as point group data). A point group is a collection of multiple three-dimensional points that represents the three-dimensional shape of an object. Point group data includes position information and attribute information of multiple three-dimensional points. The position information represents the three-dimensional position of each three-dimensional point. In addition, position information is sometimes referred to as geometry information. For example, position information is represented by an orthogonal coordinate system or a polar coordinate system.

[0083] The attribute information indicates, for example, color information, reflectance, transmittance, infrared information, normal vector, or time information, etc. One three-dimensional point may have a single attribute information or may have multiple types of attribute information.

[0084] In addition, the following mainly describes encoding and decoding of position information, but the encoding device can also encode and decode attribute information.

[0085] [Triangle collection method]

[0086] The encoding device according to the present embodiment encodes position information using a triangle set (TriSoup: Triangle-Soup, triangle patch set) method.

[0087] The triangle set method is one of the methods for encoding the position information of the point group data, and is an irreversible compression method. In the triangle set method, the element point group of the processing object is replaced with a set of triangles, and the point group is approximated on the plane. Specifically, the element point group is replaced with the vertex information in the node, and the vertices are connected to each other to generate a triangle group. In addition, the vertex information used to generate the triangle is stored in the bit stream and sent to the decoding device.

[0088] First, the encoding process using the triangle set method will be described. Figure 1 is a graph representing an example of a primitive point group. Figure 1 As shown, a point group 102 of an object is contained in an object space 101 and includes a plurality of points 103 .

[0089] First, the encoding device performs octree segmentation on the elementary point group until the specified depth. In the octree segmentation, the object space is divided into 8 nodes (subspaces), and 8 bits of information (occupancy code) are generated to indicate whether each node contains a point group. In addition, the node containing the point group is further divided into 8 nodes, and 8 bits of information are generated to indicate whether each of the 8 nodes contains a point group. This process is repeated until the predetermined level.

[0090] Here, in the conventional octree encoding, for example, the segmentation is repeated until the number of point groups included in the node becomes 1 or below a threshold. On the other hand, in the triangle set method, the octree segmentation is performed until the intermediate level, and the octree segmentation is not performed for the levels below the intermediate level. Such an octree until the intermediate level is called a pruned octree.

[0091] Figure 2 is a diagram showing an example of a pruned octree. Figure 2 As shown, the point group 102 is divided into a plurality of leaf nodes 104 (bottom-level nodes) of the pruned octree.

[0092] Next, the encoding device performs the following processing on each leaf node 104 of the pruned octree. In addition, the leaf node is also referred to as a node below. The encoding device generates a vertex on the edge as a representative point of a point group close to the edge of the node. The vertex is called an edge vertex. For example, an edge vertex is generated for each edge of a plurality of edges (e.g., four parallel edges).

[0093] Figure 3 is an example of two-dimensionally displaying leaf nodes 104, for example, Figure 1 The xy plane is viewed from the z direction as shown. Figure 3 As shown, an edge vertex 112 is generated on the edge based on the proximity points of the edge among a plurality of points 111 within the leaf node 104 .

[0094] In addition, Figure 3 In the example, the dotted line on the periphery of the leaf node 104 is an edge. In addition, in this example, at a point within a distance of 1 from an edge ( Figure 3 The edge vertex 112 is generated at the weighted average position of the positions of the points included in the range 113 in the range 114. In addition, the unit of distance is, for example, the resolution of the point group, but is not limited to this. In addition, the distance (threshold) is 1 in this example, but it can also be other than 1 and can be variable.

[0095] Next, the encoding device generates a vertex inside the node based on a point group existing in the normal direction of the plane including the plurality of edge vertices. This vertex is referred to as a centroid vertex.

[0096] Figure 4 and Figure 5 is a diagram for explaining the method of generating centroid vertices. First, the encoding device selects, for example, 4 points from the edge vertex group as representative points. Figure 4 In the example shown, edge vertices v1 to v4 are selected. Next, the encoding device calculates an approximate plane 121 passing through the four points. Next, the encoding device calculates a normal line n of the approximate plane 121 and an average coordinate M of the four points. Next, the encoding device calculates one or more points (e.g., Figure 5 The centroid vertex C is generated at the weighted average coordinates of the points included in the range 122 shown.

[0097] Next, the encoding device entropy encodes vertex information as information of edge vertices and centroid vertices, and stores the encoded vertex information in a geometry data unit (GDU) included in the bitstream. In addition, the GDU includes not only vertex information but also information indicating a pruned octree.

[0098] Figure 6 is a diagram showing an example of vertex information. Figure 6 As shown, the point group 102 is converted into vertex information 123 .

[0099] Next, the decoding process of the bit stream generated as described above is described. First, the decoding device decodes the GDU from the bit stream to obtain vertex information. Next, the decoding device connects the vertices to generate a triangle collection surface (TriSoup-Surface) as a triangle group.

[0100] Figure 7 is a diagram showing an example of a triangle set surface. Figure 7 In the example shown, four edge vertices v1 to v4 and a centroid vertex C are generated based on vertex information. In addition, a triangle 131 (triangle collection face) is generated with the centroid vertex C and two edge vertices as vertices. For example, a group of two edge vertices on two adjacent edges is selected, and a triangle 131 is generated with the selected group and the centroid vertex as vertices.

[0101] Figure 8 is a diagram for explaining the restoration process of the point group. By performing the above process on each leaf node, Figure 8 As shown, a three-dimensional model representing the object using a plurality of triangles 131 is generated.

[0102] Next, the decoding device generates points 132 at regular intervals on the surface of the triangle 131 , thereby restoring the position information of the point group 133 .

[0103] [Slice boundary processing]

[0104] The following describes the case where a point group is divided into multiple slices and encoded using a triangle set method. In such a case, if the slice width is inconsistent with the integer multiple of the leaf node width, the points are sometimes not restored at the slice boundary. Specifically, when the point group spans the adjacent first and second slices, the leaf node belonging to the first slice and spanning the first and second slices does not include the point group included in the second slice, so there is a problem of blank areas inside the node.

[0105] At the edge of a node connected to a blank area, there is no point group close to the edge due to the blank area, so an edge vertex cannot be generated on the edge. Or, when an edge vertex is generated in this state, the distance between the point group and the edge is far away, so the vertex position does not reflect the actual point group distribution, and there is a problem that the accuracy of the decoded point group is deteriorated.

[0106] Fig. 91 is a diagram showing an example in which positions extending in the vertical direction of a point cloud are divided into slices. Fig.10 is a diagram showing an example of the vertices generated in this case. Fig.10 In the example shown, the slice is divided from the bottom with nodes of a specified width. In this case, the node 1 at the top of the slice contains a blank area. Therefore, although the edge vertex v2 should be generated on the upper edge, it cannot be generated because there is no point group near the edge. In addition, Fig. 9 and Fig.10 The slice boundaries shown are the boundaries of the slice's bounding box.

[0107] Fig.11 This is a diagram showing an example of a triangle aggregate surface that should be generated. Fig.12 It means Fig.10 The figure shows an example of a triangle collection face without generating edge vertices. Fig.12 As shown, when edge vertices v2 and v3 are not generated on the upper edge, a triangle aggregate surface is generated only by vertices v1 and v4 on the lower edge and centroid vertex c. As a result, the triangle aggregate surface that should have been vertically cut in the node is generated only at the lower part of the node.

[0108] Fig.13 is a diagram showing an example of a restored point group in this case. Fig.13 As shown in , the points are not restored at the locations where there is no triangle collection face, so Fig.13 As shown in the region 201, there are holes running through the restored point group.

[0109] In this embodiment, the width of the node located at the end of the bounding box of the slice is set to a width different from the predetermined width, thereby suppressing the generation of a blank area in the node and generating edge vertices that cannot be generated in the above-mentioned problem.

[0110] Fig.14 : is a diagram showing an example of vertices generated in this case. Fig.15 is a diagram showing an example of a triangle collection surface in this case. Fig.14 As shown in FIG. 1 , a node 1 having a width outside the specified range is set at the end of the slice. In this way, the blank space in the node 1 can be eliminated and the edge vertex v2 can be generated. Fig.15 As shown, a triangle collection surface can be generated by vertically sectioning the node, thereby eliminating the situation where a transverse hole is generated in the restored point group.

[0111] In addition, the encoding device stores information indicating the slice width, for example, in the GDU header as adjustment width information for calculating the adjustment width of the node outside the specified width.

[0112] Here, the node with the width outside the prescribed range is a node whose side length along at least one of the vertical and horizontal heights is different from the prescribed length (prescribed width). The node with the width outside the prescribed range is a rectangular parallelepiped or cube shape that is different from the cube shape of the prescribed width. In addition, the adjustment width information is information for adjusting the prescribed length of the side of the node to the length of the side that is different from the prescribed width (the width outside the prescribed range). For example, the adjustment width information can represent the length of the adjustment width itself, the difference between the prescribed width and the adjustment width, or the ratio of the prescribed width to the adjustment width.

[0113] For example, the adjusted width (out-of-spec width) of an out-of-spec width node is expressed by min (slice width - node position, specified width). That is, the out-of-spec width of a node is set to the minimum value between (slice width - node position) and the specified width. Fig.10 As shown, it is the position (coordinate) of the corner closest to the origin among the corners of the node.

[0114] As described above, edge vertices can also be generated at nodes at the ends of the bounding box of the slice. This allows the triangle collection surface to be arranged without interruption, thereby eliminating the occurrence of holes in the restored point group.

[0115] In addition, as another method, for example, the following method may be used. In addition, here, the focus is on solving the problem in the encoding process and the standard.

[0116] The point group included in the first slice is also included in the leaf nodes of the second slice. In other words, there are overlapping point groups in the first slice and the second slice.

[0117] The point group is divided into slices by making the boundary coordinates between the first slice and the second slice coincide with the leaf node width. In other words, no nodes with blanks due to slice division are generated.

[0118] [syntax]

[0119] The following describes the transmission information sent from the encoding device to the decoding device in order to implement the above-mentioned method. For example, the transmission information is stored in a bit stream.

[0120] Fig.16 FIG. 4 is a diagram showing an example of the transmission information. Fig.16 As shown, the GDU header included in the bitstream includes an out-of-spec width processing flag and slice width information. The out-of-spec width processing flag is information indicating whether the above-mentioned out-of-spec width node is set. For example, a value of 1 indicates that the out-of-spec width node is set, and a value of 0 indicates that the out-of-spec width node is not set. The slice width information indicates the slice width (the width of the bounding box of the slice).

[0121] Fig.17 This is a diagram showing an example of the syntax of the GDU header (geometry_data_unit_header). Fig.17 As shown, the slice width information is stored in the GDU header only when the out-of-spec width processing flag is set to a value of 1 (true). The slice width information is composed of information indicating the width of the slice on each of the x, y, and z axes, for example.

[0122] The slice width information is information in units of slices. Here, the out-of-spec width processing flag and the slice width information are stored in the GDU header which is a header in units of slices.

[0123] Furthermore, the out-of-spec width processing flag may be information indicating whether the slice width information is included in the bitstream. In addition, the names of the flags and information shown in this embodiment are examples, and any names may be used.

[0124] Fig.18 This is a diagram showing an example of setting the adjustment width of a node with a width outside the specified range. As described above, the adjustment width is expressed by min (slice width - node position, specified width). Fig.18 An example of setting the adjustment width in the x-axis direction is shown. In this example, adjustment width = min (100-96, 32) = 4. The same calculation can be performed for the y-axis and z-axis directions.

[0125] In addition, an example in which the out-of-spec width processing mark and slice width information are stored for each slice is described here, but this information can also be common to multiple slices. In this case, this information can also be stored in a header higher (superior) than the GDU header, such as SPS or GPS. SPS (Sequence Parameter Set) is metadata (parameter set) common to multiple frames. GPS (Geometry Parameter Set) is metadata (parameter set) related to the encoding of location information. For example, GPS is metadata common to multiple frames.

[0126] In addition, a flag indicating whether the information is stored in the upper header or stored for each slice may be stored in the SPS or GPS. In this case, the storage destination of the information is switched based on the flag.

[0127] Note that this information is information used for processing unique to the triangle aggregation method, and therefore may be stored in the bit stream only when the encoding method is the triangle aggregation method.

[0128] [Processing Flow]

[0129] The following describes the flow of processing in the encoding device and the decoding device. Fig.19 This is a flowchart of the encoding process performed by the encoding device.

[0130] First, the encoding device generates a pruned octree and stores octree information representing the pruned octree in the GDU (S101). For example, the encoding device performs entropy encoding on the octree information and stores the encoded octree information in the GDU.

[0131] Next, the encoding device determines whether the slice width (the width of the slice boundary box) is an integer multiple of the specified width (S102). If the slice width is not an integer multiple of the specified width (S102: No), the encoding device stores the out-of-spec width processing flag = 1 and the slice width information in the GDU header (S103).

[0132] On the other hand, when the slice width is an integral multiple of the prescribed width ( S102 : Yes), the encoding device stores an out-of-spec width processing flag=0 in the GDU header ( S104 ).

[0133] Next, the encoding device performs the following processing of steps S105 to S109 on each of the plurality of leaf nodes of the pruned octree.

[0134] First, the encoding device determines whether the out-of-spec width processing flag = 1 (S105). When the out-of-spec width processing flag = 1 (S105: Yes), the encoding device determines whether the object node exists at the end of the slice (the end of the slice boundary box) (S106). For example, when the object node includes a slice boundary, the encoding device determines that the object node exists at the end of the slice, and when the object node does not include a slice boundary, the encoding device determines that the object node does not exist at the end of the slice.

[0135] When the target node exists at the slice end (S106: Yes), the encoding device calculates the adjustment width based on the slice width indicated by the slice width information and the node position of the target node, and sets the target node as an unspecified width node having the calculated adjustment width (S107).

[0136] On the other hand, when the out-of-spec width processing flag = 0 ( S105 : No), or when the target node does not exist at the end of the slice ( S106 : No), the encoding device sets the width of the target node to the specified width ( S108 ).

[0137] Next, the encoding device generates edge vertices on the edge of the object node and generates a centroid vertex inside the object node based on the point group distribution in the object node (S109). As described above, the loop processing for the object node ends.

[0138] After the loop processing for all leaf nodes is completed, the encoding device entropy encodes the vertex information indicating the positions of the edge vertices and centroid vertices of the plurality of leaf nodes, and stores the encoded vertex information in the GDU ( S110 ).

[0139] Next, the encoding device generates a bit stream including a GDU header and a GDU, and outputs the bit stream (S111). That is, the encoding device transmits the bit stream to the decoding device.

[0140] Fig. 20 1 is a flowchart showing the decoding process performed by the decoding device. First, the decoding device obtains the GDU header and GDU from the bit stream (S121). Next, the decoding device obtains the out-of-spec width processing flag from the GDU header and determines whether the out-of-spec width processing flag = 1 (S122).

[0141] When the out-of-spec width processing flag = 1 (S122: Yes), the decoding device obtains the slice width information from the GDU header (S123). On the other hand, when the out-of-spec width processing flag = 0 (S122: No), the decoding device does not obtain the slice width information from the GDU header.

[0142] Next, the decoding device obtains the octree information from the GDU. For example, the decoding device performs entropy decoding on the encoded octree information included in the GDU to obtain the octree information. Next, the decoding device uses the octree information to generate a leaf node group of the pruned octree (S124).

[0143] Next, the decoding device performs the following processing of steps S125 to S130 on each of the plurality of leaf nodes of the pruned octree.

[0144] First, the decoding device determines whether the out-of-spec width processing flag = 1 (S125). When the out-of-spec width processing flag = 1 (S125: Yes), the decoding device determines whether the object node exists at the end of the slice (the end of the bounding box of the slice) (S126). For example, when the object node includes a slice boundary, the decoding device determines that the object node exists at the end of the slice, and when the object node does not include a slice boundary, the decoding device determines that the object node does not exist at the end of the slice.

[0145] When the target node exists at the slice end (S126: Yes), the decoding device calculates the adjustment width based on the slice width indicated by the slice width information and the node position of the target node, and sets the target node as an unspecified width node having the calculated adjustment width (S127).

[0146] On the other hand, when the out-of-spec width processing flag is not 1 (S125: No), or when the object node does not exist at the end of the slice (S126: No), the decoding device sets the width of the object node to the specified width (S128).

[0147] Next, the decoding device obtains vertex information indicating the positions of edge vertices and centroid vertices from the GDU (S129). For example, the decoding device performs entropy decoding on the encoded vertex information included in the GDU to obtain the vertex information.

[0148] Next, the decoding device connects the plurality of vertices indicated by the vertex information to generate a triangle group (S130). As described above, the loop processing for the target node ends.

[0149] After the loop processing for all leaf nodes is completed, the decoding device generates points at a certain interval on the surfaces of multiple triangles to generate a decoding point group (S131).

[0150] [Modifications]

[0151] In the above description, it is stated that the node with the specified width is at the end of the slice ( Fig.18 The example of generating the right end of the width is shown above, but the position of generating the node outside the specified width is not limited to this. Figure 21 to Figure 24 This is a diagram showing an example of setting a node with a width outside the specified range.

[0152] For example, Fig.21 As shown in , the location of the specified width node can also be the middle of the slice. Fig. 22 As shown, the position of the node with the specified outer width may also be the front end of the slice.

[0153] In addition, if Fig.23 and Fig.24 As shown in FIG. 1 , multiple nodes with widths outside the specified range may also be set. For example, Fig.23 As shown in , two extra-width nodes may also be set at both ends of the slice. Fig.24 As shown, two nodes with unspecified widths may be set at the end of the slice and in the middle of the slice.

[0154] In this way, when a plurality of nodes having an unexpected width are set, the total of the widths of the plurality of nodes having an unexpected width may be the above-mentioned adjustment width.

[0155] Based on the above, as a calculation method for adjusting the width and information for the unspecified width node stored in the bitstream, that is, unspecified width node information, the following can be used.

[0156] like Fig.21 and Fig. 22As shown, when the node with the extra width is configured outside the end of the slice, the node information with the extra width includes: the adjustment width information indicating the adjustment width, and the insertion position information indicating the insertion position of the node with the extra width. Fig.21 In the example shown, the insertion position information indicates the bit string "4b0010". Fig. 22 In the example shown, the insertion position information indicates the bit string "4b1000." In addition, each of the plurality of bits of the bit string corresponds to a node, and bits corresponding to nodes of a width outside the specified range are set to 1, and bits other than these are set to 0.

[0157] In addition, as another method, the insertion position information may also indicate information for determining the node with the width outside the specified range. For example, the node may be assigned a consecutive number (identifier), and the insertion position information may indicate the consecutive number of the node with the width outside the specified range. For example, a consecutive number may be set for each axis in the slice. In this case, Fig.21 In the example shown, consecutive numbers 1, 2, 3, and 4 are set in order from the left node, and the insertion position information indicates a value of 3. Alternatively, the consecutive numbers may be consecutive numbers for all nodes in the slice.

[0158] In this case, the decoding device can use the insertion position information to determine whether the target node is a node outside the specified width. Fig.18 The adjustment width is calculated in the same way as in the case shown.

[0159] On the other hand, Fig.23 and Fig.24 As shown in FIG. 1 , when the extra-width node is configured at multiple positions in the slice, the insertion position information indicates the insertion positions of the multiple extra-width nodes. Fig.23 In the example shown, the insertion position information indicates the bit string "5b1001". Fig.24 In the example shown, the insertion position information represents the bit string "5b10010". In addition, although an example of using one bit string for one axis (the x-axis in this example) is shown here (that is, an example of using a different bit string for each axis), one bit string may be used for all nodes in a slice.

[0160] In addition, Fig.23 and Fig.24 In the example shown, the adjustment width information indicates the adjustment width of each of the two unspecified width nodes. Specifically, the adjustment width information indicates a value of 2 as the adjustment width of the first unspecified width node and a value of 2 as the adjustment width of the second unspecified width node.

[0161] Alternatively, the adjustment width information may also indicate the total of the adjustment widths of two out-of-spec width nodes and the adjustment width of one out-of-spec width node. In this case, the decoding device may also calculate the adjustment widths of the two out-of-spec width nodes based on the adjustment width information. In this case, for example, a rule may be pre-determined that the adjustment width information of the last out-of-spec width node in the arrangement of out-of-spec width nodes on one axis is omitted, and the decoding device calculates the adjustment widths of the two out-of-spec width nodes based on the adjustment width information based on the rule. In addition, the out-of-spec width node information may not be composed of information for each axis, but may be composed of individual information of all out-of-spec width nodes in a slice.

[0162] For example, in Fig.24 In the example shown, when the adjustment width information of the fourth node is omitted, the adjustment width of the fourth node is obtained as the total of the adjustment widths (4) - the adjustment width of the first node (2) = 2.

[0163] Additionally, the start position of the bounding box of the slice may also have an offset relative to the slice boundary. Fig.25 2 is a diagram showing an example of setting the nodes with the unspecified width in this case. Fig.25 The example shown is an example in which the node at the end of the slice is set to a node with an unspecified width.

[0164] In this case, the unspecified width node information includes an offset from the origin coordinate to the start position of the bounding box of the slice, and the decoding device can calculate the adjustment width using the offset.

[0165] Specifically, the adjustment width is obtained as min(slice width (100)-node position (121)+offset (25), specified width)=min(4,32)=4.

[0166] In addition, the decoding device may also use the octree information to determine whether the object node is located at the end of the slice. For example, by referring to the occupied code from the root node of depth = 0 in sequence for a certain coordinate axis, and continuing to move only toward the side close to the origin or away from the side, the node at the end of the slice can be determined. In this case, the specified width node information includes information indicating the mantissa node width of the adjusted width of the node as the end of the slice. The decoding device uses this information to set the adjusted width of the node at the end of the slice to the mantissa node width.

[0167] Fig.26 2 is a diagram showing an example of setting a node with an unspecified width in this case. In this example, the mantissa node width = 4, and the decoding device sets the width of the node to 4 when the target node is a node at the end of a slice, and sets the width of the node to 32 otherwise.

[0168] In addition, the unspecified width may be a value larger than the specified width. Fig.21 In the example shown, the third node and the fourth node may be combined. In other words, three nodes may be set, the widths of which are 32, 32, and 36 from the left.

[0169] Furthermore, as a result of using unspecified widths for three axes of a certain node, namely, the x-axis, the y-axis, and the z-axis, the node may also be a cube.

[0170] Alternatively, instead of the slice width information, the position information of the corners of the node that can restore the size of the node outside the specified width may be transmitted. For example, the position information may be the coordinate information of two corners in the direction outside the specified width, or the coordinate information of all eight corners.

[0171] The encoding device may also quantize the above-mentioned node information of the specified width for calculating the adjustment width, and transmit the quantized information to the decoding device. In this case, the decoding device may also inversely quantize the quantized information, and use the obtained node information of the specified width to perform the above-mentioned processing. In this case, there is a possibility that the width of the blank area in the node of the specified width does not completely become 0. On the other hand, the amount of data of the information to be transmitted can be reduced by quantization.

[0172] It is also possible to transmit information of nodes with widths outside the specified range by combining the above ideas.

[0173] In addition, the default width of the node is determined based on the size of the bounding box of the octree encoding target (the bit depth of the original data) and the depth to which the octree segmentation is performed. The default width is expressed by a power of 2, for example.

[0174] In the above description, the node with a width outside the prescribed width is a node whose length along at least one of the heights of the vertical and horizontal directions is different from the prescribed width, but the length in only one direction or only two directions may be different from the prescribed width. That is, the node with a width outside the prescribed width may also be a cuboid.

[0175] In the above description, four edge vertices existing on four parallel edges of a node are obtained, but the number of edge vertices to be obtained is not limited to 4. The number of edge vertices is not particularly limited as long as an approximate plane can be obtained.

[0176] The method of finding the centroid vertex is not limited to the above method, and the centroid vertex may be found by other methods as long as the plane of the triangle can be found by the decoding device.

[0177] In addition, in the above description, the triangle set method is used as the compression method, but the method of this embodiment is also effective for compression methods other than the triangle set method. That is, the method of this embodiment is effective for compression methods that approximate a point group by a plane or a curved surface within a node and require edge vertices to generate the plane or the curved surface.

[0178] In addition, in the above description, the length outside the prescribed width of the side of the node outside the prescribed width is obtained, but it is not necessary to obtain the length outside the prescribed width. As long as the shape of the node outside the prescribed width can be obtained, for example, the positions of the two ends of the side of the outside the prescribed length can also be obtained. In other words, the position of the node outside the prescribed width can also be obtained.

[0179] [Adjustment method on the origin side]

[0180] exist Fig.25 In FIG. 1 , it is described that the start position of the bounding box of the slice does not coincide with the origin. The following describes this situation in detail.

[0181] The origin of the slice is an unspecified offset, and the starting position of the slice's bounding box is not necessarily consistent with the origin. Therefore, there may be a situation where the point group of the slice after subtracting the offset is distributed away from the origin of the encoding coordinate system. In other words, there is a situation where the origin of the encoding coordinate system is different from the origin of the slice's bounding box. In addition, in this case, the boundary of the slice's bounding box may also be inconsistent with the boundary of the leaf node. In this case, it is necessary to generate a node of the specified width on both the side of the slice's bounding box that is close to the origin and the side of the bounding box that is away from the origin.

[0182] The encoding device transmits the slice position as the front end coordinates of the bounding box of the slice and the slice width as the width of the bounding box to the decoding device as information for the decoding device to calculate the node position and node width (adjustment width) of the above-mentioned unspecified width node.

[0183] Fig. 27 FIG. 4 is a diagram showing an example of setting a node with an unspecified width in this case. Fig. 27 In the example, the coordinate system is encoded with a one-dimensional marker. In addition, the shaded area is the area (slice) where the point group is distributed. In addition, Fig. 27 The origin shown is the origin of the encoded coordinate system.

[0184] Here, if the prescribed width of the node is set to W, the slice position is set to A, the slice width is set to B, and the original node position is set to nodePos, the adjusted node position newNodePos and the adjusted node width newNodeWidth are calculated as follows.

[0185] newNodePos=(nodePos<A)? A: nodePos;

[0186] newNodeWidth=(nodePos<A)? (W-(A-nodePos)): min(A+B-nodePos+1, W)

[0187] As described above, if nodePos<A, the node position is adjusted to A, otherwise the node position is not adjusted. That is, the node position of node 1 as the front node is changed from P1 to A, while the node positions of other nodes are not changed.

[0188] In addition, if nodePos<A, the node width is adjusted to (W-(A-nodePos)), otherwise the node width is set to min(A+B-nodePos+1, W). That is, the node width of node 1 is adjusted to W1=W-(A-nodePos)=W-(A-P1). The node width of node 2, which is the end node, is set to W2=min(A+B-nodePos+1, W)=A+B-P2+1. The node width of other nodes is set to W. Here, the reason for using "+1" is that there is a relationship of node width=width of connotation point coordinates+1. Fig.28 is a graph showing the relationship between the width of a node and the width of the coordinates of the connotation point. For example, in the case of W = 8, A = 0, B = 5, nodePos = 0, Fig.28 As shown, newNodeWidth=6.

[0189] In addition, when the out-of-spec width processing mark included in the header of the bit stream = 1, the decoding device calculates A and B based on the transmitted information, and uses the above formula to calculate the adjusted node position and adjusted node width of node 1 and node 2 based on the initial node position and initial node width of each node.

[0190] In addition, if Figure 21 to Figure 24 As shown, in this case, the position of the predetermined width node may be located at the front end of the slice, the middle of the slice, both ends of the slice, or multiple positions.

[0191] Furthermore, when nodes having an unspecified width are provided at a plurality of positions on the same axis, the total node width thereof may be the above-mentioned adjustment width.

[0192] Based on the above, as a calculation method and information for a node with an unspecified width stored in a bitstream, that is, unspecified width node information, the following can be used.

[0193] Fig.29is a diagram showing an example of setting a node with a width outside the specified range. Fig.29 As shown, in the box of the slice boundary, the starting position of the slice boundary box has an offset relative to the origin, and the slice width (95) is not a multiple of the specified width (32) of the node, if the encoding device transmits the offset from the origin to the starting position of the boundary box to the decoding device, the decoding device can calculate the adjusted position and adjusted width of node 1 on the origin side.

[0194] In addition, if the encoding device transmits the slice width to the decoding device, the decoding device can calculate the adjustment width of the node 2 on the side away from the origin.

[0195] Specifically, here, in Fig.29 When the node position of the node 1 on the origin side before adjustment is 32, the adjusted position of the node 1 = (32 < 37)? 37: 32 = 37. In addition, the adjusted width of the node 1 = (32 < 37)? (32-(37-32)): min (37+95-32+1, 32) = 27.

[0196] Adjusted position of node 2 on the side away from the origin = (128<37)? (32-(37-128)): min (37+95-128+1, 32) = 5.

[0197] In addition, the out-of-spec width node information may include information for specifying an out-of-spec width node and information for indicating an adjusted position and an adjusted width of the specified node, in addition to information for calculating based on the positional relationship between the node position and the bounding box of the slice. The information for specifying an out-of-spec width node may indicate, for example, a consecutive number assigned to the node.

[0198] Fig.30 FIG. 4 is a diagram showing an example of setting a node with an unspecified width in this case. Fig.30 In the example shown, the node numbers are assigned as consecutive numbers of 0 to 3. If the adjustment position and adjustment width of the node with node number = 0 are set to P0 and W0, and the adjustment position and adjustment width of the node with node number = 3 are set to P1 and W1, then Fig.30 In the example shown, P0=37, W0=27, P1=128, W1=5, and these information are transmitted.

[0199] In addition, Fig.29 The determination of the node with a width outside the specified range and the calculation method of its position and width are described in , but the determination method of the node with a width outside the specified range can also be used in Figure 21 to Figure 26 Any of the methods described in .

[0200] In addition, the transmission information transmitted from the encoding device to the decoding device for realizing the present embodiment is shown below. Fig.31 This is a diagram showing a syntax example of GPS (geometry_parameter_set) and the GDU header.

[0201] The GPS includes a first out-of-spec width processing flag and a second out-of-spec width processing flag. The first out-of-spec width processing flag is information indicating whether the node position and node width are adjusted for the node at the end of the slice close to the origin. For example, a value of 1 indicates that the adjustment is performed, and a value of 0 indicates that the adjustment is not performed.

[0202] The second out-of-spec width processing flag is information indicating whether the node width adjustment described above is performed for a node at a slice end located away from the origin. For example, a value of 1 indicates that the adjustment is performed, while a value of 0 indicates that the adjustment is not performed.

[0203] When the first out-of-spec width processing flag is 1, the GDU header includes first bit length information, first quantization parameter, and slice position information.

[0204] The slice position information indicates the slice position as the position (coordinates) of the bounding box of the slice, and for example, indicates the three-dimensional coordinates (x, y, z coordinates) of the corner closest to the origin among the corners of the bounding box of the slice.

[0205] The first bit length information indicates the bit length of the slice position information. The first quantization parameter indicates the quantization parameter (quantization value) used for quantization of the slice position information.

[0206] When the second out-of-spec width processing flag is 1, the GDU header includes the second bit length information, the second quantization parameter, and the slice width information.

[0207] The slice width information indicates the slice width, which is the width of the bounding box of the slice. For example, the slice width information indicates the width of the bounding box in each of the x, y, and z directions.

[0208] The second bit length information indicates the bit length of the slice width. The second quantization parameter indicates the quantization parameter used for quantization of the slice width information.

[0209] Here, the slice position is expressed by slice position information << the first quantization parameter, and the slice width is expressed by slice width information << the second quantization parameter.

[0210] In addition, an example of storing this information per slice is described here, but when this information is common in multiple slices, this information can also be stored in a header higher than the GDU header, such as SPS or GPS. In addition, a mark indicating whether this information is stored in a higher-level header or stored per slice can also be stored in SPS or GPS. In this case, based on the mark, the storage destination of this information is switched. In addition, this information can also be set individually for each axis of x, y, and z.

[0211] Alternatively, the encoding device may not transmit Fig.31 The first out-of-spec width processing flag and the second out-of-spec width processing flag are shown. In this case, when the first quantization parameter = 0, the decoding device does not adjust the node position and node width as described above for the node at the end of the slice located closer to the origin, and performs the adjustment in other cases. In addition, when the second quantization parameter = 0, the decoding device does not adjust the node width as described above for the node at the end of the slice located farther from the origin, and performs the adjustment in other cases.

[0212] In addition, since the slice position information and the slice width information are numerical values ​​of the encoding coordinate system, they can also be determined as positive values.

[0213] [Transmission and processing omitted]

[0214] In the above description, the slice width information and slice position information are always stored in the header for the point group of the coding object, and the position and width of the nodes need to be recalculated for all nodes of all slices. In practice, there is a situation where the starting end of the slice of the coding object coincides with the origin, or the ending end of the slice coincides with the ending end of the node. In this case, there is no need to adjust the node position or node width for the node at the starting end or the ending end of the slice, so the transmission of the information used for the adjustment process and the recalculation of the node position and node width can be omitted.

[0215] The encoding device determines whether to perform the omission and transmits information indicating the determination result to the decoding device. In this transmission, either a dedicated mark can be used or the first bit length information and the second bit length information mentioned above can be used. Specifically, when the first bit length information indicates 0, it means that the transmission of the slice position information is omitted, and when the second bit length information indicates 0, it means that the transmission of the slice width information is omitted. In this way, the amount of header data can be suppressed and the processing time of the encoding process can be shortened.

[0216] Fig.32This is a diagram showing an example of a slice and a node when the starting end of the slice coincides with the origin. In this case, there is no need to perform adjustment processing of the starting end.

[0217] Fig.33 This is a diagram showing an example of a slice and a node when the start end of the slice coincides with the origin and the end end of the slice coincides with the end end of the node. In this case, no adjustment process is required for both the start end and the end end.

[0218] Fig.34 This is a diagram showing an example of processing (syntax) in the case of the above omission. slice_bb_pos_bits shown in the figure is the first bit length information indicating the bit length of the slice position information. slice_bb_width_bits is the second bit length information indicating the bit length of the slice width information. In addition, A is the slice position, B is the slice width, nodePos is the node position (node ​​position before adjustment), nodeWidth is the node width (node ​​width before adjustment), newNodeWidth is the node width after adjustment, and W is the specified width of the node.

[0219] In this process, the adjustment process is limited to the case where the bit length of the slice position information is a value greater than 0, so the processing time of the encoding process can be reduced. In addition, when the starting end of the slice coincides with the origin, or the ending end happens to coincide with the ending end of the node, the amount of header data can be reduced.

[0220] Fig.35 This is a diagram showing an example of the syntax of the GPS and GDU headers in this case. Fig.35 In the syntax shown, for Fig.31 The syntax shown adds the following two conditions: the first quantization parameter and slice position information are stored in the GDU header only when the first bit length information is greater than 0; and the second quantization parameter and slice width information are stored in the GDU header only when the second bit length information is greater than 0.

[0221] By adding this condition, when the start end of the slice coincides with the origin, or the end end coincides with the end end of the node, the amount of header data can be reduced.

[0222] Fig.36 This is a flowchart of the omission determination process of the start end adjustment process (slice position transmission). The encoding device determines whether the start position of the slice (the coordinates of the start end of the slice boundary box) coincides with the origin (S201).

[0223] When the start position of the slice coincides with the origin (S201: Yes), the encoding device does not transmit the slice position information (S202). That is, the encoding device stores slice_bb_pos_bits=0 in the bitstream and does not store the slice position information in the bitstream.

[0224] On the other hand, when the start position of the slice does not coincide with the origin ( S201 : No), the encoding device transmits the slice position information ( S203 ). That is, the encoding device stores slice_bb_pos_bits set to a value greater than 0 and the slice position information in the bitstream.

[0225] The above determination may be performed only when the first out-of-spec width processing flag is 1. When the first out-of-spec width processing flag is 0, the slice position information is not transmitted.

[0226] Fig.37 This is a flowchart of the omission determination process of the end adjustment process (slice width transmission). The encoding device determines whether the end position of the slice (the coordinates of the end of the bounding box of the slice) matches the end of the node (S211).

[0227] When the end position of the slice matches the end end of the node (S211: Yes), the encoding device does not transmit the slice width information (S212). That is, the encoding device stores slice_bb_width_bits=0 in the bitstream and does not store the slice width information in the bitstream.

[0228] On the other hand, when the end position of the slice does not match the end end of the node (S211: No), the encoding device transmits the slice width information (S213). That is, the encoding device stores slice_bb_width_bits set to a value greater than 0 and the slice width information in the bitstream.

[0229] The above determination may be performed only when the second out-of-spec width processing flag is 1. When the second out-of-spec width processing flag is 0, the slice width information is not transmitted.

[0230] Fig.38 is a flowchart of the node position determination process. For example, Fig.38 The processing shown is performed for each node.

[0231] First, the encoding device determines whether the first bit length information (slice_bb_pos_bits) is greater than 0 ( S221 ).

[0232] When the first bit length information is greater than 0 ( S221 : Yes), the encoding device determines whether the node position (nodePos) of the target node is smaller than the slice position ( S222 ).

[0233] When the node position is smaller than the slice position ( S222 : Yes), the encoding device sets the slice position to the adjusted node position ( S223 ).

[0234] On the other hand, when the first bit length information is 0 (S221: No), or when the node position is greater than or equal to the slice position (S222: No), the encoding device does not change (adjust) the node position (S224).

[0235] Furthermore, the same processing is performed in the decoding device.

[0236] Fig.39 is a flowchart of the node width determination process. For example, Fig.39 The processing shown is performed for each node.

[0237] First, the encoding device determines whether the first bit length information (slice_bb_pos_bits) is greater than 0 ( S231 ).

[0238] When the first bit length information is greater than 0 ( S231 : Yes), the encoding device determines whether the node position (nodePos) of the target node is smaller than the slice position ( S232 ).

[0239] When the node position is smaller than the slice position ( S232 : Yes), the encoding device sets the predetermined width (W)-(slice position (A)-node position (nodePos)) as the adjusted node width (nodeWidth) ( S233 ).

[0240] On the other hand, when the node position is greater than or equal to the slice position ( S232 : No), the encoding device determines whether the second bit length information (slice_bb_width_bits) is greater than 0 ( S234 ).

[0241] When the second bit length information is greater than 0 (S234: Yes), the encoding device sets min(slice position (A)+slice width (B)-node position (nodePos)+1, specified width (W)) as the node width (nodeWidth) (S235).

[0242] On the other hand, when the second bit length information is 0 (S234: No), the encoding device does not change the node width (nodeWidth) (S236). In other words, the encoding device sets the node width to a predetermined width (W).

[0243] In addition, when the first bit length information is 0 (S231: No), the encoding device determines whether the second bit length information (slice_bb_width_bits) is greater than 0 (S237). In addition, when the first bit length information is 0 (S231: No), the slice position information is not transmitted, and the start end of the slice coincides with the origin.

[0244] When the second bit length information is greater than 0 (S237: Yes), the encoding device sets min(slice width (B)-node position (nodePos)+1, prescribed width (W)) as the node width (nodeWidth) (S238).

[0245] On the other hand, when the second bit length information is 0 (S237: No), the encoding device does not change the node width (nodeWidth) (S236). In other words, the encoding device sets the node width to a predetermined width (W).

[0246] [Summary]

[0247] As described above, the decoding device (three-dimensional data decoding device) according to the embodiment performs Fig.40 The processing shown. The decoding device is a decoding device that decodes multiple three-dimensional points. The decoding device obtains multiple nodes having an octree structure and contained in the first slice from the bit stream (S301), obtains information for deriving the shape of the first node among the multiple nodes from the bit stream (S302), and decodes the first node according to the information (S303), and the shape of the first node is different from the prescribed shapes of other nodes among the multiple nodes. Thus, a node with a shape different from the prescribed shape can be set. Thus, a variable node can be set according to the size of the slice or the distribution of the point group. Thus, there is a possibility of improving the coding efficiency.

[0248] For example, the shape of the first node is a cuboid, not a cube. For example, the end of the first slice is consistent with the end of a node among the multiple nodes. Thus, in the case where the node end is inconsistent with the slice end, the node end can be made consistent with the slice end. Thus, the situation where no vertex is generated at the node end can be suppressed, so the generation of a blank area at the slice boundary can be suppressed. Thus, the accuracy of the decoded point group can be improved.

[0249] For example, the size of the shape of the first node is different from the predetermined size of the predetermined shape. For example, the length of the side of the shape of the first node is different from the predetermined length (eg, predetermined width) of the side of the predetermined shape.

[0250] For example, the information for deriving the shape of the first node indicates the size of the shape of the first node or the positions of both ends of the side of the first node. Thus, the decoding device can generate a node having a shape different from a predetermined shape using this information.

[0251] For example, the information for deriving the shape of the first node includes adjustment information (e.g., slice width information or slice position information) for adjusting the prescribed shape to the shape of the first node. Thus, the decoding device can generate a node having a shape different from the prescribed shape using the adjustment information. In addition, there is a possibility that the amount of information can be reduced compared to the case where the absolute amount of information of the position is transmitted.

[0252] For example, the decoding of the first node is performed according to a compression method that approximates a plurality of three-dimensional points by a plane or a curved surface within the first node. For example, the compression method is a Triangle-Soup compression method.

[0253] For example, the shape of the first node is determined in order to generate a plane or a curved surface in the first node. Thus, by setting a node with a shape different from the predetermined shape, a plane or a curved surface can be generated in the first node.

[0254] For example, the edge of the shape of the first node has a vertex, and the plane or curved surface intersects the edge at the vertex. Thus, by setting a node of a shape different from the prescribed shape, a plane or curved surface can be generated within the first node. For example, the plurality of three-dimensional points includes the first three-dimensional point located near the vertex.

[0255] For example, the first node is set in contact with the second slice adjacent to the first slice. Thus, for example, when the node end is inconsistent with the slice end, it is possible to suppress the situation where the vertex is not generated at the node end, so that the blank area at the slice boundary can be suppressed. For example, when only nodes of a specified shape are set at the slice boundary, there is a situation where the node is divided by the slice boundary. In this case, the point group in the second slice adjacent to the first slice cannot be used to encode or decode the first node, so there is a possibility that the restoration accuracy of the three-dimensional point group near the slice boundary will deteriorate. Therefore, in this method, by setting a node with a shape different from the specified shape, for example, the end of the first node can be made consistent with the end of the first slice. Thus, it is possible to suppress the restoration accuracy of the three-dimensional point group near the slice boundary. In addition, if this method is applied to the triangle set method, it is possible to suppress the situation where the edge vertices are not properly generated.

[0256] For example, information for deriving the shape of the first node is set for each slice, and information for the second slice is used to derive the shape of the second node among the plurality of nodes having an octree structure and included in the second slice, and the shape of the second node is different from the prescribed shape. Thus, the shape of the node can be set for each slice.

[0257] For example, the size of a predetermined shape is expressed by a power of 2, and the size of the shape of the first node is different from the size expressed by the power of 2.

[0258] For example, the shape of the first node is defined by a first length along a first direction, a second length along a second direction, and a third length along a third direction, the first direction, the second direction, and the third direction are orthogonal to each other, and only the first length among the first length, the second length, and the third length is different from the length of the prescribed shape of nodes other than the first node, or, only the first length and the second length among the first length, the second length, and the third length are each different from the length of the prescribed shape.

[0259] For example, the first node is closest to the origin of the first slice among the plurality of nodes in one direction among the first direction, the second direction, and the third direction, and the first direction, the second direction, and the third direction are orthogonal to each other. Thus, when the starting position of the slice is inconsistent with the origin, for example, the starting position of the node can be adjusted in accordance with the starting position of the slice. In addition, the origin is a position that serves as a reference for defining the position or shape of a slice, a node, or a three-dimensional point.

[0260] For example, the plurality of nodes includes a third node having a shape different from the prescribed shape, and the third node is farthest from the origin in one direction among the plurality of nodes. Thus, when the end position of the slice does not coincide with the end position of the node, the end position of the node can be adjusted in accordance with the end position of the slice, for example.

[0261] For example, when the starting position of the first slice is inconsistent with the origin, the bitstream includes information for deriving the shape of the first node, and when the starting position of the first slice is consistent with the origin, the bitstream does not include information for deriving the shape of the first node. Thus, the occurrence of processing for adjusting the shape of the node can be reduced. In addition, the transmission of information for deriving the shape of the node can be omitted. Thus, the amount of processing can be reduced and the amount of data in the bitstream can be reduced. In addition, the starting position (starting end) of the slice is the position of the end of the slice on the side close to the origin, and the ending position (ending end) of the slice is the position of the end of the slice on the side away from the origin. Similarly, the starting position (starting end) of the node is the position of the end of the node on the side close to the origin, and the ending position (ending end) of the node is the position of the end of the node on the side away from the origin.

[0262] For example, when the end position of the first slice is inconsistent with the end end of the first node, the bitstream includes information for deriving the shape of the first node, and when the end position of the first slice is consistent with the end end of the first node, the bitstream does not include information for deriving the shape of the first node. Thus, the occurrence of processing for adjusting the shape of the node can be reduced. In addition, the transmission of information for deriving the shape of the node can be omitted. Thus, the amount of processing can be reduced and the amount of data in the bitstream can be reduced.

[0263] Fig.41 It is a module of the decoding device 10. For example, the decoding device 10 includes a processor 11 and a memory 12, and the processor 11 uses the memory 12 to perform the above-mentioned processing.

[0264] In addition, the encoding device (three-dimensional data encoding device) involved in the embodiment performs Fig.42 The processing shown. The encoding device is an encoding device that encodes multiple three-dimensional points. The encoding device encodes multiple nodes having an octree structure and contained in the first slice, thereby generating a bit stream (S311), and stores information for deriving the shape of the first node among the multiple nodes in the bit stream (S312), and the shape of the first node is different from the prescribed shapes of other nodes among the multiple nodes. Thus, a node with a shape different from the prescribed shape can be set. Thus, a variable node can be set according to the size of the slice or the distribution of the point group. Thus, there is a possibility of improving the coding efficiency. In addition, the encoding device can also perform the same processing as the above-mentioned decoding device.

[0265] Fig.43 It is a module of the encoding device 20. For example, the encoding device 20 includes a processor 21 and a memory 22, and the processor 21 uses the memory 22 to perform the above-mentioned processing.

[0266] As mentioned above, the encoding device (three-dimensional data encoding device) and the decoding device (three-dimensional data decoding device) etc. which concern on embodiment and modification of this disclosure are demonstrated, but this disclosure is not limited to this embodiment.

[0267] In addition, each processing unit included in the encoding device and decoding device according to the above-mentioned embodiments is typically implemented by an LSI which is an integrated circuit. They may be individually single-chip or partially or entirely single-chip.

[0268] Furthermore, the integrated circuit is not limited to LSI, and can be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI can also be used.

[0269] In addition, in each of the above-mentioned embodiments, each component may be formed by dedicated hardware, or implemented by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

[0270] In addition, the present disclosure can also be implemented as an encoding method (three-dimensional data encoding method) or a decoding method (three-dimensional data decoding method) executed by an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device).

[0271] In addition, the present disclosure may also be implemented as a program that enables a computer, a processor, or a device to perform the above-mentioned encoding method or decoding method. In addition, the present disclosure may also be implemented as a bit stream generated by the above-mentioned encoding method. In addition, the present disclosure may also be implemented as a recording medium having the program or the bit stream recorded thereon. For example, the present disclosure may also be implemented as a non-volatile computer-readable recording medium having the program or the bit stream recorded thereon.

[0272] In addition, the division of the functional modules in the block diagram is an example, and multiple functional modules can also be implemented as one functional module, or one functional module can be divided into multiple modules, or a part of the function can be transferred to other functional modules. In addition, the functions of multiple functional modules with similar functions can also be processed in parallel or in time-sharing by a single hardware or software.

[0273] In addition, the order in which each step in the flowchart is executed is an example order for specifically explaining the present disclosure, and may be an order other than the above. In addition, part of the above steps may be executed simultaneously (in parallel) with other steps.

[0274] The above describes one or more encoding devices and decoding devices involved in the embodiments, and the present disclosure is not limited to the embodiments. As long as it does not deviate from the main purpose of the present disclosure, the embodiments obtained by applying various modifications thought of by those skilled in the art to the present embodiment, and the embodiments constructed by combining the constituent elements in different embodiments can also be included in the scope of one or more embodiments.

[0275] Industrial Applicability

[0276] The present disclosure can be applied to encoding devices and decoding devices.

[0277] Description of reference numerals:

[0278] 10 Decoding device

[0279] 11, 21 Processor

[0280] 12.22 Memory

[0281] 20 Encoding device

[0282] 101 Object Space

[0283] 102, 133 point groups

[0284] 103, 111, 132 points

[0285] 104 leaf nodes

[0286] 112 edge vertices

[0287] 113, 122 Range

[0288] 121 Approximate plane

[0289] 123 Vertex Information

[0290] 131 Triangle

[0291] 201 Area

Claims

1. A decoding method for decoding multiple three-dimensional points. From the bitstream, obtain a plurality of nodes having an octree structure and included in the first slice, obtaining information for deriving a shape of a first node among the plurality of nodes from the bit stream, decoding the first node according to the information, The shape is different from prescribed shapes of other nodes among the plurality of nodes.

2. The decoding method according to claim 1, The shape is a cuboid shape, not a cube shape.

3. The decoding method according to claim 1, An end of the first slice coincides with an end of any one of the plurality of nodes.

4. The decoding method according to claim 1, The information indicates the size of the shape or the positions of both ends of the side of the first node.

5. The decoding method according to claim 1, The information includes adjustment information for adjusting the prescribed shape to the shape.

6. The decoding method according to claim 1, The decoding is performed according to a compression method of approximating the plurality of three-dimensional points by a plane or a curved surface within the first node.

7. The decoding method according to claim 6, The compression method is a Triangle-Soup compression method, that is, a triangle set compression method.

8. The decoding method according to claim 6, The shape is determined in order to generate the plane or the curved surface in the first node.

9. The decoding method according to claim 8, having vertices on the sides of the shape, The plane or the curved surface intersects the edge at the vertex.

10. The decoding method according to claim 1, The first node is disposed in contact with a second slice adjacent to the first slice.

11. The decoding method according to claim 10, The information is set per slice. The information of the second slice is used to derive a shape of a second node having an octree structure and among a plurality of nodes included in the second slice. The shape of the second node is different from the predetermined shape.

12. The decoding method according to claim 1, The size of the predetermined shape is expressed by a power of 2, and the size of the shape is different from the size expressed by a power of 2.

13. The decoding method according to claim 1, The shape of the first node is defined by a first length along a first direction, a second length along a second direction, and a third length along a third direction, the first direction, the second direction, and the third direction being orthogonal to each other, Among the first length, the second length and the third length, only the first length is different from the length of the prescribed shape of the node other than the first node, or, among the first length, the second length and the third length, only the first length and the second length are each different from the length of the prescribed shape.

14. The decoding method according to claim 1, The first node is closest to the origin of the first slice among the plurality of nodes in one direction among a first direction, a second direction, and a third direction, and the first direction, the second direction, and the third direction are orthogonal to each other.

15. The decoding method according to claim 14, The plurality of nodes include a third node having a shape different from the prescribed shape, The third node is, among the plurality of nodes, the farthest from the origin in the one direction.

16. The decoding method according to claim 1, In the case where the start position of the first slice is not consistent with the origin, the bit stream includes the information, When the start position of the first slice coincides with the origin, the bitstream does not include the information.

17. The decoding method according to claim 1, In the case where the end position of the first slice is inconsistent with the end end of the first node, the bit stream includes the information, When the end position of the first slice coincides with the end end of the first node, the bitstream does not include the information.

18. A coding method for encoding a plurality of three-dimensional points. Encoding a plurality of nodes having an octree structure and included in the first slice to generate a bit stream, storing information for deriving the shape of a first node among the plurality of nodes in the bit stream, The shape is different from prescribed shapes of other nodes among the plurality of nodes.

19. A decoding device for decoding a plurality of three-dimensional points, comprising: Processor; and Memory, The processor uses the memory, From the bitstream, obtain a plurality of nodes having an octree structure and included in the first slice, obtaining information for deriving a shape of a first node among the plurality of nodes from the bit stream, decoding the first node according to the information, The shape is different from prescribed shapes of other nodes among the plurality of nodes.

20. A coding device for encoding a plurality of three-dimensional points, comprising: Processor; and Memory, The processor uses the memory, Encoding a plurality of nodes having an octree structure and included in the first slice to generate a bit stream, storing information for deriving the shape of a first node among the plurality of nodes in the bit stream, The shape is different from prescribed shapes of other nodes among the plurality of nodes.

Images