Decoding method, encoding method, decoding device, and encoding device

Abstract

This decoding method involves deriving a plurality of sorted edge vertices of a node by trisouping (S301), deriving a face vertex placed on a node surface excluding a plurality of sides of the node (S302), selecting two adjacent edge vertices adjacent to each other in the sorting order and closest to the face vertex among the plurality of sorted edge vertices (S303), and generating a trisoup triangle by using the face vertex and one of the two adjacent edge vertices (S304).

Description

Decoding method, encoding method, decoding device, and encoding device

[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device.

[0002] In the future, devices and services that utilize 3D data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, video distribution, etc. 3D data can be acquired in a variety of ways, including distance sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

[0003] One method of representing three-dimensional data is a representation called a point cloud, which represents the shape of a three-dimensional structure using a group of points in three-dimensional space. In a point cloud, the position and color of the points are stored. Point clouds are expected to become the mainstream method of representing three-dimensional data, but point clouds have a very large amount of data. Therefore, when storing or transmitting three-dimensional data, it is essential to compress the amount of data by encoding, just as with two-dimensional moving images (examples include MPEG-4 AVC or HEVC standardized by MPEG).

[0004] Furthermore, compression of point clouds is partially supported by a public library (Point Cloud Library) that performs processing related to point clouds.

[0005] Furthermore, a technique is known in which three-dimensional map data is used to search for and display facilities located around a vehicle (see, for example, Patent Document 1).

[0006] International Publication No. 2014 / 020663

[0007] In such encoding and decoding of three-dimensional data, it is desirable to be able to improve the accuracy of the reconstructed three-dimensional data.

[0008] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve the accuracy of reconstructed three-dimensional data.

[0009] A decoding method according to one aspect of the present disclosure derives a sorted plurality of edge vertices of a node according to the Trisorp method, derives face vertices that are located on the face of the node excluding the plurality of edges of the node, selects two adjacent edge vertices from the sorted plurality of edge vertices that are adjacent to each other in sorted order and are closest to the face vertex, and generates a Trisorp triangle using one of the two adjacent edge vertices and the face vertex.

[0010] An encoding method according to one aspect of the present disclosure derives a sorted number of edge vertices of a node according to the Trithorpe method, derives face vertices that are located on the face of the node excluding the number of edges of the node, selects two adjacent edge vertices from the sorted number of edge vertices that are adjacent to each other in sorted order and are closest to the face vertex, and generates a Trithorpe triangle using one of the two adjacent edge vertices and the face vertex.

[0011] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve the accuracy of reconstructed three-dimensional data.

[0012] FIG. 1 is a diagram illustrating an example of an original point cloud according to an embodiment. FIG. 2 is a diagram illustrating an example of a pruned octree according to an embodiment. FIG. 3 is a diagram illustrating an example of a two-dimensional display of leaf nodes according to an embodiment. FIG. 4 is a diagram illustrating a method for generating centroid vertices according to an embodiment. FIG. 5 is a diagram illustrating a method for generating centroid vertices according to an embodiment. FIG. 6 is a diagram illustrating example vertex information according to an embodiment. FIG. 7 is a diagram illustrating an example of a Trithorpe surface according to an embodiment. FIG. 8 is a diagram illustrating a point cloud restoration process according to an embodiment. FIG. 9 is a diagram illustrating an example of a point cloud according to an embodiment. FIG. 10 is a diagram illustrating an example of centroid vertex generation according to an embodiment. FIG. 11 is a diagram illustrating an example of a triangle (Trithorpe surface) generation according to an embodiment. FIG. 12 is a diagram illustrating an example of face vertex generation according to an embodiment. FIG. 13 is a diagram illustrating an example of a surface for evaluating connectivity and restoring centroid vertices according to an embodiment. FIG. 14 is a flowchart of encoding processing according to an embodiment. FIG. 15 is a flowchart of decoding processing according to an embodiment. FIG. 16 is a flowchart of transmission processing of face vertex information according to an embodiment. FIG. 17 is a flowchart of a process for decoding face vertex information according to an embodiment. FIG. 18 is a diagram showing examples of face vertex candidates according to an embodiment. FIG. 19 is a diagram showing an example of an arrangement of a vertex group according to an embodiment. FIG. 20 is a diagram showing a vector NF according to an embodiment. FIG. 21 is a diagram showing an example of an arrangement of a vertex group according to an embodiment. FIG. 22 is a diagram showing an example of an arrangement of a vertex group according to an embodiment. FIG. 23 is a diagram showing an example of an arrangement of a vertex group according to an embodiment. FIG. 24 is a diagram showing examples of edge vertices and centroid vertices according to an embodiment. FIG. 25 is a diagram showing an example of a correctly generated triangle according to an embodiment. FIG. 26 is a diagram showing an example of an erroneously generated triangle according to an embodiment. FIG. 27 is a diagram showing examples of edge vertices, centroid vertices, and face vertices according to an embodiment. FIG. 28 is a diagram showing an example of triangle generation when simplified ordering according to an embodiment is applied. FIG. 29 is a diagram showing example of edge vertices after simplified sorting according to an embodiment.Fig. 30 is a diagram showing an example of edge vertices and face vertices after ordering according to an embodiment. Fig. 31 is a diagram showing an example of generated triangles according to an embodiment. Fig. 32 is a diagram showing an example of GDU syntax according to an embodiment. Fig. 33 is a diagram showing a modified example of GDU syntax according to an embodiment. Fig. 34 is a diagram showing a modified example of GDU syntax according to an embodiment. Fig. 35 is a flowchart of decoding processing according to an embodiment. Fig. 36 is a block diagram of a decoding device according to an embodiment. Fig. 37 is a flowchart of encoding processing according to an embodiment. Fig. 38 is a block diagram of an encoding device according to an embodiment.

[0013] A decoding method according to one aspect of the present disclosure derives a sorted plurality of edge vertices of a node according to the Trisorp method, derives face vertices that are located on the face of the node excluding the plurality of edges of the node, selects two adjacent edge vertices from the sorted plurality of edge vertices that are adjacent to each other in sorted order and are closest to the face vertex, and generates a Trisorp triangle using one of the two adjacent edge vertices and the face vertex.

[0014] This allows the decoding method to appropriately select combinations of vertices used to generate Trithorpe triangles from the edge vertices and face vertices when face vertices are used in addition to edge vertices, thereby improving the accuracy of the generated Trithorpe triangles and the accuracy of the restored 3D data.

[0015] For example, the order of the face vertices in the sort order may be set between the orders of the two adjacent edge vertices, so that the decoding method can appropriately select a combination of vertices to be used to generate a Trithorpe triangle according to the set order.

[0016] For example, in deriving the sorted edge vertices, the edge vertices of the node and the face vertices of the node may be projected onto a plane having a normal in a predetermined direction, and the edge vertices may be sorted based on the arctangent values ​​of vectors connecting each of the edge vertices projected onto the plane with the centroid vertex. In this way, the decoding method can appropriately sort the edge vertices.

[0017] For example, the derivation of the sorted edge vertices may include projecting the edge vertices of the node and the centroid vertices of the node onto a plane normalized in a predetermined direction, and if the centroid vertices projected onto the plane include a first centroid vertex projected onto an edge of the node on the plane, sorting the first vertices including the edge vertices and the first centroid vertex, selecting two adjacent edge vertices from the sorted first vertices that are adjacent to each other in sorting order and are closest to a second face vertex from the centroid vertices projected onto a portion other than the edge of the node on the plane, and generating a trisorp triangle using one of the two adjacent edge vertices and the second face vertex. This simplifies the sorting process for some face vertices, thereby reducing the amount of processing.

[0018] For example, in selecting the two adjacent edge vertices, the midpoints of pairs of edge vertices that are adjacent to each other in the sorting order among the sorted edge vertices may be calculated, and the edge vertex pair corresponding to the midpoint that is closest to the face vertex among the calculated midpoints may be selected as the two adjacent edge vertices. In this way, the decoding method can appropriately select the two adjacent edge vertices that are closest to the face vertex.

[0019] An encoding method according to one aspect of the present disclosure derives a sorted number of edge vertices of a node according to the Trithorpe method, derives face vertices that are located on the face of the node excluding the number of edges of the node, selects two adjacent edge vertices from the sorted number of edge vertices that are adjacent to each other in sorted order and are closest to the face vertex, and generates a Trithorpe triangle using one of the two adjacent edge vertices and the face vertex.

[0020] This allows the encoding method to appropriately select combinations of vertices used to generate Trithorpe triangles from the edge vertices and face vertices when face vertices are used in addition to edge vertices, thereby improving the accuracy of the generated Trithorpe triangles and the accuracy of the restored 3D data.

[0021] For example, the order of the face vertices in the sort order may be set between the orders of the two adjacent edge vertices, and the set order of the edge vertices and the face vertices may be used to encode attribute information of the 3D points. In this way, the encoding method can appropriately encode the attribute information by encoding the attribute information of the 3D points using the set order.

[0022] In addition, a decoding device according to one aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to derive a plurality of sorted edge vertices of a node according to a Trisorp method, derive face vertices to be located on the face of the node excluding a plurality of edges of the node, select two adjacent edge vertices from the sorted plurality of edge vertices that are adjacent to each other in sorted order and are closest to the face vertex, and generate a Trisorp triangle using one of the two adjacent edge vertices and the face vertex.

[0023] In addition, an encoding device according to one aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to derive a plurality of sorted edge vertices of a node according to a Trithorpe method, derive face vertices to be located on the face of the node excluding a plurality of edges of the node, select two adjacent edge vertices from the sorted plurality of edge vertices that are adjacent to each other in sorting order and are closest to the face vertex, and generate a Trithorpe triangle using one of the two adjacent edge vertices and the face vertex.

[0024] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0025] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0026] (Embodiment) Hereinafter, an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device) according to this embodiment will be described. The encoding device generates a bitstream by encoding three-dimensional data. The decoding device generates three-dimensional data by decoding the bitstream.

[0027] The three-dimensional data is, for example, three-dimensional point cloud data (also called point cloud data). A point cloud is a collection of multiple three-dimensional points and represents the three-dimensional shape of an object. The point cloud data includes position information and attribute information of multiple three-dimensional points. The position information indicates the three-dimensional position of each three-dimensional point. Note that the position information may also be called geometry information. For example, the position information is expressed in a Cartesian coordinate system or a polar coordinate system.

[0028] The attribute information indicates, for example, color information, reflectance, transmittance, infrared information, normal vector, time information, etc. One 3D point may have a single piece of attribute information or may have multiple types of attribute information.

[0029] Although the following mainly describes the encoding and decoding of position information, the encoding device may also encode and decode attribute information.

[0030] [TriSoup Method] The encoding device according to this embodiment encodes position information using the TriSoup (Triangle-Soup) method.

[0031] The Trithorpe method is a lossy compression method that encodes the positional information of point cloud data. The Trithorpe method converts the original point cloud to be processed into a set of triangles, and then approximates the point cloud on that plane. Specifically, the original point cloud is converted into vertex information within a node, and the vertices are connected to generate a set of triangles. The vertex information used to generate the triangles is then stored in a bitstream and sent to a decoding device.

[0032] First, the encoding process using the Trithorpe method will be described. Fig. 1 shows an example of an original point cloud. As shown in Fig. 1, a point cloud 102 of an object is contained in an object space 101 and includes a plurality of points 103.

[0033] First, the encoding device divides the source point cloud into an octree to a predetermined depth. In the octree division, the target space is divided into eight nodes (subspaces), and 8-bit information (occupancy code) indicating whether or not a point cloud is included in each node is generated. Furthermore, the node including the point cloud is further divided into eight nodes, and 8-bit information indicating whether or not a point cloud is included in each of the eight nodes is generated. This process is repeated up to a predetermined hierarchy.

[0034] In normal octree coding, division is repeated until the number of points included in a node is reduced to one or a threshold value. On the other hand, in the Trithorpe method, octree division is performed up to an intermediate layer, but not to layers below that layer. Such an octree up to an intermediate layer is called a pruned octree.

[0035] 2 is a diagram showing an example of a pruned octree. As shown in FIG. 2, a point cloud 102 is divided into a plurality of leaf nodes 104 (nodes at the bottom level) of the pruned octree.

[0036] Next, the encoding device performs the following process on each leaf node 104 of the pruned octree. Note that hereinafter, a leaf node may also be simply referred to as a node. The encoding device generates a vertex on the edge as a representative point of a group of points close to the edge of the node. This vertex is called an edge vertex. For example, an edge vertex is generated for each of multiple edges (e.g., four parallel sides).

[0037] 3 is an example of a two-dimensional display of the leaf node 104, for example, a diagram showing the xy plane as viewed from the z direction shown in Fig. 1. As shown in Fig. 3, an edge vertex 112 is generated on the edge based on the nearest points of the edge among the multiple points 111 in the leaf node 104.

[0038] In Fig. 3, the dotted line around the periphery of the leaf node 104 is the edge. In this example, the edge vertex 112 is generated at the weighted average position of the points (points included in the range 113 in Fig. 3) whose distance from the edge is within 1. The unit of distance is, for example, the resolution of the point cloud, but is not limited to this. In this example, this distance (threshold) is 1, but it may be a value other than 1 or may be variable.

[0039] Next, the encoding device generates a vertex inside the node based on a group of points existing in the normal direction of the plane containing the edge vertices. This vertex is called a centroid vertex.

[0040] 4 and 5 are diagrams illustrating a method for generating a centroid vertex. First, the encoding device selects, for example, four points as representative points from a group of edge vertices. In the example shown in FIG. 4, edge vertices v1 to v4 are selected. Next, the encoding device calculates an approximate plane 121 that passes through the four points. Next, the encoding device calculates a normal n to the approximate plane 121 and an average coordinate M of the four points. Next, the encoding device generates a centroid vertex C at the weighted average coordinate of one or more points (for example, points included in the range 122 shown in FIG. 5) that are close to a half-line extending from the average coordinate M in the direction of the normal n.

[0041] Next, the encoding device entropy-encodes the vertex information, which is information on the edge vertices and centroid vertices, and stores the encoded vertex information in a Geometry Data Unit (GDU) included in the bitstream. Note that the GDU includes information indicating the pruned octree in addition to the vertex information.

[0042] 6 is a diagram showing an example of vertex information. Through the above processing, the point group 102 is converted into vertex information 123 as shown in FIG.

[0043] Next, the decoding process of the bitstream generated above will be described. First, the decoding device decodes the GDU from the bitstream to obtain vertex information. Next, the decoding device connects the vertices to generate a TriSoup-Surface, which is a group of triangles.

[0044] 7 is a diagram showing an example of a Trithorpe surface. In the example shown in FIG. 7, four edge vertices v1 to v4 and a centroid vertex C are generated based on the vertex information. In addition, a triangle 131 (Trithorpe surface) is generated whose vertices are the centroid vertex C and two edge vertices. For example, pairs of two edge vertices on two adjacent edges are selected, and a triangle 131 is generated whose vertices are the selected pairs and the centroid vertex.

[0045] 8 is a diagram for explaining the point cloud restoration process. By performing the above process for each leaf node, a three-dimensional model is generated in which the target object is represented by a plurality of triangles 131, as shown in FIG.

[0046] Next, the decoding device generates points 132 at regular intervals on the surface of the triangle 131 to restore the position information of the point group 133 .

[0047] [Example of Representation of Ridge Lines of Point Cloud Surfaces] In some cases, the Trithorpe method cannot reconstruct the shape of ridge lines (edge ​​lines) that cross adjacent nodes. In response to this, the encoding device generates face vertices on the faces that abut adjacent nodes, and reconstructs point clouds on the triangular faces generated from the centroid vertices, face vertices, and edge vertices.

[0048] For example, if there are bends in the point cloud distribution (point cloud surface) inside a leaf node, the corners and edges of the point cloud surface do not intersect, and no vertices are created at the corner positions, so the surface model created by connecting the vertices may not be able to reproduce the shape of the original point cloud.

[0049] 9 is a diagram showing an example of a point cloud in which a ridge line is formed by distributing the point cloud across nodes 1 and 2. As shown in FIG. 9, an edge vertex 112 is generated based on the point cloud distribution near the edge.

[0050] 10 is a diagram showing an example of generating a centroid vertex in this case. As shown in Fig. 10, a centroid vertex 151 is generated in the normal direction of the approximation plane of the edge vertex group.

[0051] Fig. 11 shows an example of the generation of a triangle 131 (Trithorpe surface) in this case. As shown in Fig. 11, the triangle 131 is generated by connecting multiple vertices (multiple edge vertices and centroid vertices). In this case, as shown in Fig. 11, the point cloud near the node boundary cannot be reproduced.

[0052] This is because the centroid vertices can sample the original point cloud surface, but the current method cannot create a vertex between two centroid vertices of two adjacent nodes. For example, if a ridge line continues to be distributed within a node along one of the x, y, or z axes, a vertex corresponding to the ridge line will not be generated because the ridge line does not cross an edge. This causes this problem.

[0053] In this embodiment, the encoding device predicts the ridge line of the point cloud surface, and if it determines that two adjacent nodes have the same ridge line, it transmits information for connecting the two centroid vertices of the two adjacent nodes with a line segment to the decoding device. This information is, for example, one bit of information assigned to each face between the nodes.

[0054] The decoder uses this information to connect the centroid vertices and generate new vertices (face vertices) at the intersections of the resulting line segments and the shared faces between the nodes. The decoder can use these new vertices to reconstruct the ridge lines when generating triangles 131.

[0055] Furthermore, since the coordinate positions of the face vertices are not quantized, there is no problem of positional deviation due to quantization.

[0056] 12 is a diagram showing an example of generating face vertices. As shown in Fig. 12, the decoding device generates face vertices 161 and generates triangles 131 using the face vertices 161, thereby reproducing the ridge line.

[0057] The above method can reproduce the point cloud surface near the node boundary, resulting in a decoded point cloud that is closer to the original point cloud. Note that in the above explanation, the point cloud surface is used only to explain the issues related to ridge lines, and the ridge lines do not need to be actually obtained.

[0058] [Overview of Generation, Transmission, and Restoration of Face Vertices] First, we will explain the evaluation and restoration of connectivity between centroid vertices. For each node, the encoding device generates a line segment connecting the centroid vertex of the target node to the centroid vertex of the adjacent node, and determines the connectivity between the centroid vertices from the weight of the point cloud near the intersection of the line segment and the shared face between the nodes.

[0059] The encoding device sets a Boolean value (for example, the face vertex information described above) indicating whether or not to connect two centroid vertices for the corresponding face to generate a face vertex.

[0060] The Boolean value of each face is transmitted from the encoding device to the decoding device, and for faces with a Boolean value = true, the decoding device generates a face vertex at the position where the face intersects with the line segment connecting the centroid vertices of the nodes on both sides of the face.

[0061] 13 is a diagram showing an example of faces, among the six faces of a node, for which connectivity evaluation and restoration of centroid vertices are performed. For example, as shown in FIG. 13, the encoding device and decoding device perform connectivity evaluation and restoration of centroid vertices on both sides of the faces for the three faces with larger coordinates on each axis among the six faces of the node.

[0062] Next, we will explain the outline of reducing the data amount of the bitstream and limiting the face information. To suppress the amount of data to be transmitted, the encoding device applies conditions to the face vertex information (a set of Boolean values) using information already known to the decoding device, thereby reducing the data amount of the face vertex information to be transmitted. Details of this process will be described later.

[0063] Next, we will explain the ordering of vertices within a node. To generate a Trithorpe surface, it is necessary to appropriately select two edge vertices, or an edge vertex and a face vertex. For example, if an edge vertex close to a face vertex is not selected, and an edge vertex far from the face vertex is selected, the surface will not approximate the original point group. Furthermore, an approximate face will not be generated. Therefore, in order to create a surface without gaps or holes within the node, for example, it is necessary to order the edge vertices and face vertices using a rotation order based on the centroid vertex. This will be described in detail later.

[0064] [Processing Flow] Fig. 14 is a flowchart of the encoding process by the encoding device. First, the encoding device generates a pruned octree by dividing a point cloud into octrees, and generates multiple leaf nodes (leaf node group) of the pruned octree. The encoding device then arithmetically encodes (entropy encodes) octree information indicating the pruned octree, and stores the encoded octree information in a bitstream (S201).

[0065] Next, for each of the multiple nodes (leaf nodes), the encoding device generates edge vertices and centroid vertices from the distribution of point clouds within the node, arithmetically encodes (entropy encodes) the vertex information indicating each node's position information, and stores the encoded vertex information in a bitstream (S202).

[0066] Next, the encoding device generates a face vertex at the position where the line segment connecting the centroid vertex of the target node and the centroid vertex of the adjacent node intersects with the face, only for the faces among the multiple faces of each node that meet the geometric conditions (S203).

[0067] Next, the encoding device encodes the face vertex information of the faces that meet the geometry conditions and stores the encoded face vertex information in the bitstream (S204). Here, the face vertex information is information that indicates whether or not to generate a face vertex by connecting the centroids of the nodes on both sides of the face.

[0068] Next, the encoding device performs the following processing (loop processing) of steps S205 to S208 for each of the leaf nodes of the pruned octree. First, the encoding device orders the edge vertices and face vertices in the node counterclockwise (S205). Next, the encoding device connects the vertices in the node (the edge vertices, centroid vertices, and face vertices) to generate a triangle (a trithorpe surface) (S206).

[0069] Next, the encoding device generates multiple points on the surface of the triangle (S207). Next, the encoding device makes the restored point group within the node unique using coordinate values ​​and adds them to the decoded point group (S208). Here, making them unique means excluding points with duplicate coordinate values. This completes the loop processing for the target node.

[0070] 15 is a flowchart of the decoding process performed by the decoding device. First, the decoding device arithmetically decodes (entropy decodes) octet information from the bitstream, generates a pruned octet using the octet information, and generates a plurality of leaf nodes (leaf node group) (S211).

[0071] Next, the decoding device arithmetically decodes vertex information indicating the positions of edge vertices and centroid vertices from the bitstream (S212).

[0072] Next, the decoding device arithmetically decodes face vertex information only for faces that meet the geometry conditions among the multiple faces of each leaf node (S213). Next, the decoding device generates face vertices based on the face vertex information (S214).

[0073] Next, the decoding device performs the following processing (loop processing) of steps S215 to S218 for each of the leaf nodes of the pruned octree. First, the decoding device orders the edge vertices and face vertices in the node counterclockwise (S215). Next, the decoding device generates a triangle (a trithorpe surface) by connecting the vertices in the node (the edge vertices, the centroid vertex, and the face vertices) (S216).

[0074] Next, the decoding device generates multiple points on the surface of the triangle (S217). Next, the decoding device makes the restored point group within the node unique using coordinate values ​​and adds them to the decoded point group (S218). Here, making them unique means excluding points with duplicate coordinate values. This completes the loop processing for the target node.

[0075] FIG. 16 is a flowchart showing the face vertex information transmission process (details of steps S203 and S204 shown in FIG. 14).

[0076] The encoding device performs the following processing (loop processing) of steps S221 to S226 for each of the multiple faces of each node. First, the encoding device determines whether the target face to be processed satisfies a first condition for generating face vertices (S221). Note that the first condition is a restrictive condition based on geometry information for reducing the data volume of the bitstream described below, and will be described in detail later. Furthermore, by setting this condition, it is possible to exclude faces for which face vertices cannot be generated based on the positional relationships between the node, edge vertices, and centroid vertices, thereby reducing the amount of information transmitted.

[0077] If the first condition is satisfied (Yes in S221), the encoding device determines whether the target surface satisfies a second condition for generating face vertices (S222). The second condition refers to an evaluation of the weight of points near the positions of face vertex candidates in the connectivity evaluation of centroid vertices, which will be described later. This will be described in detail later. By setting this condition, face vertices can be generated based on the distribution of the ridge line shape of the points on the surface.

[0078] If the second condition is met (Yes in S222), the encoding device sets the face vertex information of the target face to "true (vertices present)" and stores the face vertex information for transmission (S223). On the other hand, if the second condition is not met (No in S222), the encoding device sets the face vertex information of the target face to "false (vertices absent)" and stores the face vertex information for transmission (S224).

[0079] If the first condition is not met (No in S221), the encoding device does not generate face vertex information for the target surface, treats it as "false", and does not store the face vertex information for transmission (S225).

[0080] Next, the encoding device generates face vertices on the target surface based on the face vertex information (true / false) (S226). That is, if the face vertex information is "true," the encoding device generates face vertices on the target surface, and if the face vertex information is "false" (or is considered to be "false"), the encoding device does not generate face vertices on the target surface. This completes the loop processing for the target surface.

[0081] Next, the encoding device encodes the accumulated pieces of face vertex information and stores the encoded face vertex information in a bitstream (S227).

[0082] FIG. 17 is a flowchart showing the process of decoding face vertex information (details of steps S213 and S214 shown in FIG. 15).

[0083] The decoding device performs the following processing (loop processing) of steps S231 to S234 for each of the multiple faces of each node. First, the decoding device determines whether the target face satisfies a first condition for generating face vertices (S231). Note that the first condition is the same as the first condition in step S221 shown in FIG. 16.

[0084] If the first condition is satisfied (Yes in S231), the decoding device decodes face vertex information indicating whether or not to generate face vertices on the target surface from the bitstream (S232), thereby determining whether or not to generate face vertices on the target surface ("true" or "false").

[0085] Furthermore, if the first condition is not satisfied (No in S231), the decoding device does not decode the face vertex information of the target surface from the bitstream, and sets the face vertex information of the target surface to "false" (S233).

[0086] Next, the decoding device generates face vertices on the target surface based on the face vertex information (true / false) (S234). That is, if the face vertex information is "true", the decoding device generates face vertices on the target surface, and if the face vertex information is "false", the decoding device does not generate face vertices on the target surface. This completes the loop processing for the target surface.

[0087] In this way, by setting the first condition (a condition that is predetermined by the encoding device and the decoding device and cannot be changed) before the second condition of whether or not to generate a face vertex, the transmission of flag information can be suppressed.

[0088] Furthermore, by combining a predetermined first condition (a condition that cannot be changed) with a second condition (a condition that can be changed) that can be flexibly set in the encoding device and notified by a flag, it is possible to achieve both a reduction in the amount of data and flexibility in the settings.

[0089] [Evaluating and Restoring Connectivity of Centroid Vertices] The encoding device evaluates the weight of points on the line segments connecting the centroid vertices, and if the weight of points near a face vertex candidate is equal to or greater than a threshold, the candidate is set as a face vertex. Here, a candidate is the intersection of the line segment connecting the centroid vertices and the face. The weight of points near a face vertex candidate is the number or density of points included in an area within a predetermined distance from the face vertex candidate.

[0090] 18 is a diagram showing an example of face vertex candidates. In this example, the intersection of a line segment L1 connecting the centroid vertex C1 of node 1 and the centroid vertex C2 of node 2 with the shared face F1 is determined as the face vertex candidate.

[0091] The encoding device adds one bit of information (centroid vertex information) to the bit stream as transmission information to the decoding device, which indicates whether or not a vertex is to be generated on the surface by connecting the centroid vertices of the nodes on both sides of the surface. The encoding device also generates this one bit of information for each of the entire surfaces and stores the generated multiple pieces of one bit information in the bit stream.

[0092] The decoder obtains face vertex information in addition to node position information, edge vertex and centroid vertex position information, and generates face vertices for each face based on the corresponding face vertex information.

[0093] Here, the encoding device sets the intersection of the centroid vertex's line segment and the surface as the position of the face vertex. However, the face vertex may be generated at a position shifted from the intersection, for example, based on the distribution of the point cloud. In this case, in addition to the one-bit information, the encoding device stores an offset amount on the surface between the intersection and the position of the face vertex in the bitstream. This offset amount is expressed, for example, as a two-dimensional value. In other words, information indicating the position of the face vertex may be stored in the bitstream. Note that the information indicating the position of the face vertex is not limited to the offset amount, and may be coordinate information, or a coordinate difference or vector from another vertex (edge ​​vertex or centroid vertex).

[0094] This method allows the positions of the face vertices to better reflect the shape of the ridge lines of the point cloud while maintaining the connectivity of the point cloud surface between nodes, resulting in a high-quality restored point cloud.

[0095] [Reducing the amount of data in the bitstream] The encoding device sets conditions for all faces using information known to the decoding device, thereby reducing the amount of face vertex information to be transmitted. Specifically, since the decoding device knows the position information of edge vertices and centroid vertices, it uses this information to exclude pairs of centroid vertices that cannot be connected based on their geometric relationships.

[0096] For example, the encoding device limits the generation of face vertex information by ANDing the following five conditions (a) to (e): Note that the encoding device may use some of these conditions, or may further combine other conditions.

[0097] (a) The target node has a centroid vertex (C0). (b) There is a node next to the target node (there is an adjacent node). Note that it is also possible for there to be an adjacent node in either the x, y, or z axis direction. (c) The adjacent node has a centroid vertex (C1). This condition is set because a centroid vertex may not be generated if there are few edge vertices. (d) The number of edge vertices on the shared surface shared between nodes is 2 or 3. This condition assumes that a point cloud exists in the form of a ridge line between the nodes.

[0098] (e) The surface resulting from the generation of face vertices is larger than the original surface. Here, the surface is a surface composed of multiple triangles (triangle surface). Specifically, if the following three vectors are not inverted (the inner product of vector Cvec0 and vector NF and the inner product of vector Cvec1 and vector NF are both positive): (1) vector Cvec0 from the center of gravity G0 of the edge vertices of the target node to centroid vertex C0; (2) vector Cvec1 from the center of gravity G1 of the edge vertices of the adjacent node to centroid vertex C1; and (3) vector NF when the foot of the perpendicular line drawn from face vertex candidate F to the line segment formed by the two edge vertices on the shared surface is N), the face vertex information of the target surface is set as a transmission target.

[0099] Fig. 19 is a diagram showing an example of the arrangement of vertices from which face vertices are generated. Fig. 20 is a diagram showing a vector NF. In the example shown in Fig. 19, there are two edge vertices on a shared surface, and the surface expands as face vertices are generated, and face vertex information is transmitted.

[0100] 20 is a unit vector of vector E2E3. x=E2F·u, NF=E2F−xu. If Cvev0·NF>0 and Cvev1·NF>0, the surface expands.

[0101] 21 shows an example of the arrangement of vertices where two edge vertices exist on a shared surface and the surface does not bulge. In this case, face vertex information is not transmitted.

[0102] 22 shows an example of the arrangement of vertices where two edge vertices exist on a shared surface and the surface does not bulge. In this case, face vertex information is not transmitted.

[0103] 23 shows an example of the arrangement of a group of vertices where three edge vertices exist on a shared surface and the surface bulges. In this case, face vertex information is transmitted. In the example shown in FIG. 23, the pair of edge vertices (E2, E3) that is closest to the midpoint of the line segment formed by the two adjacent edge vertices E2, E3, and E4 is selected, and point N is determined.

[0104] It should be noted that the above determination does not necessarily have to be made. For example, in the case shown in Fig. 22, it may be determined that the point cloud surface bulges.

[0105] As another condition, the encoding device may determine that face vertices are to be generated in the target node if a certain number or more of face vertices have been generated consecutively in a preceding restored node connected to an adjacent node during the reconstruction process.In addition, the encoding device may determine that the point cloud surface is expanding by actually calculating the volume of the point cloud surface, rather than by using the sign of the dot product of the above-mentioned vectors, and determining that the surface is expanding if the volume increases.

[0106] The above determination corresponds to the first condition in step S221 shown in Fig. 16 and step S231 shown in Fig. 17. Also, although an example in which the encoding device makes the determination has been described here, a similar determination is also made in the decoding device.

[0107] [Ordering vertices within a node] Multiple triangles (trisorp surfaces) are generated within a node for point cloud reconstruction. In this process, the vertices must be selected in order from the edge to avoid missing any triangles. Specifically, the decoding device orders the edge vertices and face vertices in a rotational order, centered on the centroid vertex. The decoding device selects two points at a time based on the set order, and generates a triangle using the selected two points and the centroid vertex. This allows triangles to be generated without gaps within the node.

[0108] However, existing methods use the premise that the ordering targets are only edge vertices on the node frame, project the vertices from the major axis (either x, y, or z axis), and achieve ordering through simple sorting. In this embodiment, face vertices are generated on the face of the node, so the ordering targets are no longer limited to the node frame (edge), and simple sorting no longer works.

[0109] 24 to 28 are diagrams for explaining this problem. Fig. 24 is a diagram showing examples of edge vertices and centroid vertices. Fig. 25 is a diagram showing an example of a correctly generated triangle. Fig. 26 is a diagram showing an example of an erroneously generated triangle.

[0110] For example, in the example shown in Figure 24, edge vertices are ordered. For example, the decoding device projects the edge vertices in the z-axis direction to perform the ordering. That is, the decoding device orders the edge vertices in a predetermined rotation direction (clockwise or counterclockwise) around the centroid vertex on a projection plane (the x-y plane in this example) onto which the edge vertices and centroid vertices are projected. For example, the decoding device calculates the angle between a reference direction (e.g., the x-axis direction) on the projection plane and a vector from the centroid vertex to each edge vertex, calculates the arctangent of each angle, and then orders the edge vertices in the order (ascending or descending) of the obtained arctangents.

[0111] Next, the decoding device generates a triangle using a pair of the centroid vertex and two adjacent edge vertices, as shown in Fig. 25. The decoding device may select a projection direction for projecting the edge vertices from among the x-axis direction, the y-axis direction, and the z-axis direction using the following method. The decoding device projects, orders, and generates triangles for multiple edge vertices in each of the x-axis direction, the y-axis direction, and the z-axis direction, and selects, as the projection direction, the axis direction that maximizes the total area of ​​the generated triangles from the projections in the x-axis direction, the y-axis direction, and the z-axis direction. The method for determining the axis direction for projection is not limited to this.

[0112] On the other hand, as shown in FIG. 26, if the ordering of edge vertices is incorrect, holes will appear in the nodes.

[0113] Fig. 27 is a diagram showing examples of edge vertices, centroid vertices, and face vertices, and Fig. 28 is a diagram showing an example of triangle generation when simple sorting is applied in this case.

[0114] For example, when simple sorting is applied to a group of vertices including face vertices, the ordering shown in Figure 27 is performed. Note that the numbers in parentheses in Figure 27 indicate the order set by the ordering. In this case, a hole is created as shown in Figure 28. In this example, the order of face vertex F2 should be set to (5), but it has been set to (7), causing a hole.

[0115] In contrast, in this embodiment, the decoding device performs a simple sort of edge vertices (the ordering of edge vertices described above), then selects the two consecutively numbered edge vertices closest to the face vertex, and inserts the order of those face vertices between the orders of the selected two consecutively numbered edge vertices, thereby ordering multiple edge vertices and face vertices.

[0116] 29 to 31 are diagrams for explaining this processing. First, the decoding device orders the multiple edge vertices (E0 to E3) by sorting them counterclockwise using the simple sorting described above. That is, the multiple edge vertices are ordered in the order of edge vertices E0, E1, E2, and E3. FIG. 29 is a diagram showing an example of the multiple edge vertices E0 to E3 of the target node after this simple sorting has been performed.

[0117] Next, the decoding device inserts the order of face vertex F into the order of these edge vertices. First, the decoding device determines the two edge vertices closest to face vertex F. That is, the decoding device determines, among the multiple edge vertices E0 to E4, a pair of edge vertices that are adjacent to each other in the ordered order and that are closest to face vertex F. Specifically, the decoding device calculates the coordinates (three-dimensional coordinates) of the midpoints of each of multiple pairs of edge vertices that are adjacent in the order of the multiple edge vertices (in this example, the pair of E0 and E1, the pair of E1 and E2, the pair of E3 and E4, and the pair of E4 and E0). Next, the decoding device calculates the distance (distance in three-dimensional space) between each of the calculated midpoints and face vertex F, and determines the pair with the shortest calculated distance to be the pair of edge vertices closest to face vertex F.

[0118] 29, the pair of edge vertices E1 and E2 is determined to be the closest pair to face vertex F. Next, the decoding device sets the order of face vertex F between the order of the determined pair of edge vertices E1 and E2.

[0119] As explained above in [Reducing the Data Amount of Bitstream], when calculating the vector NF, the pair of edge vertices that are closest to the face vertex F are determined in the calculation. In this case, instead of performing the midpoint calculation described above, the decoding device may determine that the pair of edge vertices determined when calculating the vector NF is the pair of edge vertices that are closest to the face vertex F.

[0120] These processes are repeated for each face vertex. FIG. 30 shows an example of edge vertices and face vertices after this ordering process. As shown in FIG. 30, this ordering allows the edge vertices and centroid vertices to be properly sorted. Note that the numbers in parentheses in FIG. 30 indicate the order set by the ordering. For example, E1(2) shown in FIG. 30 indicates that the order after simple sorting is "1", but the order after the above ordering process is "2".

[0121] Next, the decoding device generates triangles using the centroid vertex and pairs of vertices (edge ​​vertices or face vertices) adjacent to each other in the ordering order. This process is repeated for each pair of vertices adjacent to each other in the ordering order, thereby generating multiple triangles. Figure 31 is a diagram showing an example of triangles generated in the ordering order. As shown in Figure 31, triangles are generated appropriately using this ordering.

[0122] The decoding device may project the face vertices in the above-described projection direction and perform simple sorting on the face vertices located on the node frame (the edges of the node in the projection plane) in the projection plane, in the same manner as for the edge vertices. That is, the decoding device performs simple sorting on a plurality of first vertices including one or more first face vertices located on the node frame in the projection plane and a plurality of edge vertices. The decoding device performs the above-described ordering process on each of one or more second face vertices not located on the node frame in the projection plane. That is, the decoding device determines, among pairs of adjacent first vertices in the ordered order of the plurality of first vertices, the pair closest to the second face vertex, and sets the order of the second face vertex between the determined orders of the pair of first vertices.

[0123] The above-described ordering process and triangle generation process may be performed in an encoding device as a point cloud reconstruction process. Furthermore, the position information of the reconstructed 3D points may be used in the encoding process of attribute information of the 3D points. Furthermore, the order of the edge vertices and face vertices set above may be used in the encoding process of the attribute information. For example, the encoding order (decoding order) of attribute information of multiple vertices and / or multiple 3D points may be set based on the order. Alternatively, the vertices or 3D points to be referenced during predictive encoding of the attribute information may be determined based on the order.

[0124] [Syntax] In relation to reducing the amount of data in the bitstream, the information transmitted from the encoding device to the decoding device for restoring face vertices is one bit of information (face vertex information) indicating whether or not to generate a face vertex on each face that is restricted based on geometry.

[0125] 32 is a diagram showing an example of the syntax of GDU (geometry_data_unit_data) included in the bitstream. The GDU includes octree information and geometry_trisoup_data. The geometry_trisoup_data includes the number of edges, edge vertex information, the number of edge vertices, edge vertex position information, the number of transmission faces, and face vertex information.

[0126] The octree information is information that indicates the configuration of a pruned octree, and is information that indicates the positions of multiple leaf nodes included in the pruned octree.

[0127] The number of edges indicates the number of unique edges. A unique edge is a unique edge that excludes edges with overlapping coordinates. Edge vertex information is provided for each edge, and edge vertex information [i] indicates whether an edge vertex exists on the i-th edge. For example, a value of 0 indicates that no edge vertex exists, and a value of 1 indicates that an edge vertex exists.

[0128] The number of edge vertices indicates the number of vertices on an edge, i.e., the number of edge vertices. Edge vertex position information is provided for each edge vertex, and edge vertex position information [i] indicates the position of the i-th edge vertex.

[0129] The number of transmitted faces indicates the total number of face information (face vertex information) to be transmitted. Face vertex information is provided for each face, and face vertex information [i] is 1-bit information indicating whether a face vertex is generated (whether a face vertex exists) on the i-th face. For example, a value of 0 indicates that a face vertex is not generated, and a value of 1 indicates that a face vertex is generated.

[0130] In addition, the number of transmission faces and face vertex information are included in the bit stream when the face vertex function is enabled, and are not included in the bit stream when the face vertex function is not enabled. The face vertex function is a process for generating the face vertices described above.

[0131] For example, a flag indicating whether the face vertex function is enabled or disabled is provided, and whether the face vertex function is enabled or disabled is determined based on this flag. This flag may be stored in, for example, the GPS or GDU header.

[0132] Furthermore, whether or not this face vertex function is enabled may be set for each node. In this case, multiple flags corresponding to multiple nodes may be stored in the GDU header.

[0133] Fig. 33 is a diagram showing a modified example of the syntax of the GDU. The syntax shown in Fig. 33 differs from the syntax shown in Fig. 32 in that the GDU includes face vertex group information instead of the number of transmission faces and face vertex information.

[0134] The face vertex group information indicates, for each of a plurality of faces, whether or not to generate face vertices on that face. In other words, the face vertex group information is information that compiles the plurality of pieces of face vertex information shown in Fig. 32. Furthermore, the face vertex group information is included in the bitstream when the face vertex function is enabled, and is not included in the bitstream when the face vertex function is not enabled.

[0135] In the syntax shown in Figure 33, the GDU does not include the number of transmission planes. Here, the node information, edge vertex position information, and centroid vertex position information for applying geometry-based restrictions to each plane are known to the decoding device. Therefore, the decoding device can calculate the number of plane information actually transmitted (the number of transmission planes) by applying geometry-based restrictions to all planes determined from the node adjacency relationships.

[0136] Fig. 34 is a diagram showing a modified example of the syntax of the GDU. The syntax shown in Fig. 34 is an example of syntax when, as described above, a face vertex is generated at a position shifted from the intersection of the face and the line segment connecting two centroid vertices. In the syntax shown in Fig. 34, the GDU further includes an offset amount (x) and an offset amount (y) compared to the syntax shown in Fig. 32.

[0137] An offset amount (x) and an offset amount (y) are set for each face vertex. An offset amount (x) [i] indicates the offset amount in the x-axis direction between the intersection point and the i-th face vertex. An offset amount (y) [i] indicates the offset amount in the y-axis direction between the intersection point and the i-th face vertex. In other words, an offset amount (x) and an offset amount (y) indicate the two-dimensional offset amount from the intersection point to the face vertex.

[0138] For example, the encoding device may quantize the two-dimensional offset amount and store it in the bitstream. In this case, the bitstream includes a quantization parameter used for quantization. The decoding device uses the quantization parameter to dequantize the quantized offset amount included in the bitstream and restore the original offset amount.

[0139] 17, the decoding device generates face vertices when the received face vertex information is true, but face vertices may be generated without relying on the face vertex information. For example, the encoding device may not store the face vertex information in the bitstream, and the decoding device may generate face vertices when a first condition is satisfied and may not generate face vertices when the first condition is not satisfied.

[0140] Also, in the flowchart shown in Figure 16, the encoding device determines the value (true or false) of the face vertex information using the first condition and the second condition, but it may also determine the value of the face vertex information using only one of the first condition and the second condition.

[0141] In the ordering of vertices within a node, the rotation axis passing through the centroid vertex is calculated from the cross product of the normal vector of the plane consisting of the edge vertices and an arbitrary coordinate axis, but the method of calculating the rotation axis for ordering vertices (multiple edge vertices and face vertices) is not limited to this. For example, the vertices may be projected in the direction of an arbitrary axis passing through the centroid vertex, and the axis direction may be determined so that the minimum value of the distance between the projection point and the axis is greater than a predetermined value. Alternatively, the axis direction may be determined so that the sum of the squares of the multiple distances is greater than a predetermined value.

[0142] Furthermore, although a Boolean value (1-bit face vertex information) is transmitted to reduce the data amount of the bitstream, other types of information may be transmitted. For example, the face vertex information may indicate three or more values. For example, a value of 0 may mean "do not generate face vertices," a value of 1 may mean "generate face vertices," and a value of 2 may mean "generate face vertices according to the capabilities of the decoding device." Alternatively, when a Boolean value is used and the Boolean value is true, the decoding device may determine whether to generate face vertices according to the capabilities of the decoding device.

[0143] [Summary] As described above, the decoding device (three-dimensional data decoding device) according to the embodiment performs the processing shown in Fig. 35. The decoding device derives sorted edge vertices of a node according to the Trithorpe method (S301), derives face vertices located on the face of the node, excluding the nodes' edges (S302), selects two adjacent edge vertices from the sorted edge vertices that are adjacent to each other in sorted order and closest to the face vertex (S303), and generates a Trithorpe triangle using one of the two adjacent edge vertices and the face vertex (S304). For example, the decoding device generates a Trithorpe triangle using one of the two adjacent edge vertices, the face vertex, and the centroid vertex, and generates a Trithorpe triangle using the other of the two adjacent edge vertices, the face vertex, and the centroid vertex.

[0144] This allows the decoding device to appropriately select combinations of vertices used to generate Trithorpe triangles from the edge vertices and face vertices when face vertices are used in addition to edge vertices, thereby improving the accuracy of the generated Trithorpe triangles and the accuracy of the reconstructed 3D data.

[0145] The decoding device may generate a trisorp triangle using a face vertex and a single edge vertex closest to the face vertex. For example, the decoding device may generate a trisorp triangle using the single edge vertex, the face vertex, and a centroid vertex. As described above, the method of selecting two adjacent edge vertices can generate multiple trisorp triangles without omissions, compared to the method of selecting only the single closest edge vertex, making it more predictable when a trisorp triangle is missing. Therefore, using the method of selecting two adjacent edge vertices can improve the accuracy of restoring the original point cloud compared to the method of selecting only the single closest edge vertex.

[0146] For example, the order of face vertices in the sort order is set between the orders of two adjacent edge vertices. For example, the decoder generates a trisorp triangle using adjacent vertex pairs and centroid vertices in the set orders of face vertices and edge vertices.

[0147] This allows the decoding device to appropriately select the combination of vertices used to generate the Trithorpe triangles in accordance with the set order.

[0148] For example, in deriving the sorted edge vertices (S301), the decoding device projects the edge vertices of the node and the face vertices of the node onto a plane having a normal in a predetermined direction, and sorts the edge vertices based on the arctangent values ​​of the vectors connecting each of the edge vertices projected onto the plane with the centroid vertex. This allows the decoding device to appropriately sort the edge vertices.

[0149] For example, in deriving the sorted edge vertices (S301), the decoding device projects the edge vertices of the node and the centroid vertices of the node onto a plane whose normal is a predetermined direction, and if the centroid vertices projected onto the plane include a first centroid vertex projected onto an edge of the node on the plane, sorts the first vertices including the edge vertices and the first centroid vertex, selects two adjacent edge vertices from the sorted first vertices that are adjacent to each other in sorted order and are closest to a second face vertex from the centroid vertices projected onto a surface other than the edge of the node on the plane, and generates a trisobe triangle using one of the two adjacent edge vertices and the second face vertex.

[0150] This allows the decoding device to simplify the sorting process for some face vertices, thereby reducing the amount of processing.

[0151] For example, in selecting two adjacent edge vertices (S303), the decoding device calculates the midpoint of each of a plurality of pairs of edge vertices that are adjacent to each other in the sorted order among the sorted plurality of edge vertices, and selects, as two adjacent edge vertices, the edge vertex pair corresponding to the midpoint that is closest to the face vertex among the calculated midpoints. This allows the decoding device to appropriately select the two adjacent edge vertices that are closest to the face vertex.

[0152] For example, in deriving sorted edge vertices (S301), the edge vertices of the node are sorted, and the face vertices of the node are not sorted.

[0153] 36 is a block diagram of the decoding device 10. For example, the decoding device 10 includes a processor 11 and a memory 12, and the processor 11 performs the above-described processing using the memory 12.

[0154] An encoding device (three-dimensional data encoding device) according to an embodiment performs the process shown in Fig. 37. The encoding device derives sorted edge vertices of a node according to the Trithorpe method (S311), derives face vertices located on the face of the node, excluding the edges of the node (S312), selects two adjacent edge vertices from the sorted edge vertices that are adjacent to each other in sorted order and closest to the face vertex (S313), and generates a Trithorpe triangle using one of the two adjacent edge vertices and the face vertex (S314). For example, the encoding device generates a Trithorpe triangle using one of the two adjacent edge vertices, the face vertex, and the centroid vertex, and generates a Trithorpe triangle using the other of the two adjacent edge vertices, the face vertex, and the centroid vertex.

[0155] This allows the encoding device to appropriately select combinations of vertices used to generate Trithorpe triangles from the edge vertices and face vertices when face vertices are used in addition to edge vertices, thereby improving the accuracy of the generated Trithorpe triangles and the accuracy of the reconstructed 3D data.

[0156] For example, the order of a face vertex in the sort order is set between the orders of two adjacent edge vertices, and the set order of the edge vertices and face vertices is used to encode the attribute information of the 3D points. In this way, the encoding device can appropriately encode the attribute information by encoding the attribute information of the 3D points using the set order.

[0157] 38 is a block diagram of the encoding device 20. For example, the encoding device 20 includes a processor 21 and a memory 22, and the processor 21 performs the above-described processing using the memory 22. Note that the encoding device may perform the same processing as the above-described decoding device.

[0158] The above describes the encoding device (three-dimensional data encoding device) and decoding device (three-dimensional data decoding device) according to the embodiments and modifications of the present disclosure, but the present disclosure is not limited to these embodiments.

[0159] Furthermore, each processing unit included in the encoding device, decoding device, etc. according to the above-described embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0160] Furthermore, the integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI may also be used.

[0161] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0162] Furthermore, the present disclosure may be realized as an encoding method (three-dimensional data encoding method) or a decoding method (three-dimensional data decoding method) executed by an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device), etc.

[0163] The present disclosure may also be realized as a program that causes a computer, processor, or device to execute the encoding method or decoding method. The present disclosure may also be realized as a bitstream generated by the encoding method. The present disclosure may also be realized as a recording medium on which the program or the bitstream is recorded. For example, the present disclosure may also be realized as a non-transitory computer-readable recording medium on which the program or the bitstream is recorded.

[0164] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0165] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0166] While the encoding device and decoding device according to one or more aspects have been described based on the embodiments, the present disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the present embodiments and configurations constructed by combining components of different embodiments may also be included within the scope of one or more aspects.

[0167] The present disclosure is applicable to encoding devices and decoding devices.

[0168] 10 Decoding device 11, 21 Processor 12, 22 Memory 20 Encoding device 101 Object space 102, 133 Point cloud 103, 111, 132 Point 104 Leaf node 112 Edge vertex 113, 122 Range 121 Approximation plane 123 Vertex information 131 Triangle 151 Centroid vertex 161 Face vertex

Claims

1. A decoding method comprising the steps of: deriving sorted edge vertices of a node according to a Trithorpe method; deriving face vertices that are located on the face of the node excluding the multiple edges of the node; selecting two adjacent edge vertices from the sorted edge vertices that are adjacent to each other in sorted order and are closest to the face vertex; and generating a Trithorpe triangle using one of the two adjacent edge vertices and the face vertex.

2. The method of claim 1, wherein the order of the face vertex in the sort order is set between the orders of the two adjacent edge vertices.

3. The decoding method according to claim 1, wherein in deriving the sorted edge vertices, the edge vertices of the node and the face vertices of the node are projected onto a plane having a predetermined direction as a normal, and the edge vertices are sorted based on the arctangent values ​​of a plurality of vectors connecting each of the edge vertices projected onto the plane with a centroid vertex.

4. The decoding method according to claim 1, wherein in deriving the sorted edge vertices, the edge vertices of the node and the centroid vertices of the node are projected onto a plane having a normal in a predetermined direction, and if a first centroid vertex projected onto an edge of the node in the plane is included among the centroid vertices projected onto the plane, sorting the first vertices including the edge vertices and the first centroid vertex, selecting two adjacent edge vertices from the sorted first vertices that are adjacent to each other in sorting order and are closest to a second face vertex from the centroid vertices projected onto a surface other than the edge of the node in the plane, and generating a trisorp triangle using one of the two adjacent edge vertices and the second face vertex.

5. The decoding method according to claim 1, wherein the selection of the two adjacent edge vertices comprises calculating a midpoint of each of a plurality of edge vertex pairs that are adjacent to each other in the sorting order among the sorted plurality of edge vertices, and selecting, as the two adjacent edge vertices, the edge vertex pair corresponding to the midpoint that is closest to the face vertex among the calculated midpoints.

6. An encoding method comprising: deriving sorted edge vertices of a node according to a Trithorpe method; deriving face vertices that are located on the face of the node excluding the multiple edges of the node; selecting two adjacent edge vertices from the sorted edge vertices that are adjacent to each other in sorted order and are closest to the face vertex; and generating a Trithorpe triangle using one of the two adjacent edge vertices and the face vertex.

7. The encoding method according to claim 6, wherein the order of the face vertices in the sorting order is set between the orders of the two adjacent edge vertices, and the set orders of the edge vertices and the face vertices are used to encode attribute information of a three-dimensional point.

8. A decoding device comprising: a processor; and a memory, wherein the processor uses the memory to: derive a sorted number of edge vertices of a node according to a Trisorpe scheme; derive face vertices to be located on the face of the node excluding a number of edges of the node; select two adjacent edge vertices from the sorted number of edge vertices that are adjacent to each other in sorted order and are closest to the face vertex; and generate a Trisorpe triangle using one of the two adjacent edge vertices and the face vertex.

9. An encoding device comprising: a processor; and a memory, wherein the processor uses the memory to: derive a plurality of sorted edge vertices of a node according to a Trithorpe scheme; derive face vertices to be located on a face of the node excluding a plurality of edges of the node; select two adjacent edge vertices from the sorted plurality of edge vertices that are adjacent to each other in sorted order and are closest to the face vertex; and generate a Trithorpe triangle using one of the two adjacent edge vertices and the face vertex.

Images