Decoding method, encoding method, decoding device, and encoding device

Abstract

The decoding method decodes a plurality of three-dimensional points, generates a first vertex at a position other than an edge of a first node on a first surface of the first node (S301), generates a plurality of second vertices at a plurality of edges of the first node (S302), generates a third vertex in the first node on the basis of the plurality of second vertices (S303), and generates a third vertex in the first node on the basis of the third vertex (S303). A triangle defined by the first vertex, a second vertex among the plurality of second vertices, and a third vertex is generated (S304), and a plurality of three-dimensional points are generated on the surface of the triangle (S305).

Description

Technical Field

[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device. Background Art

[0002] Devices and services that effectively utilize three-dimensional data are expected to become more common in a wide range of fields, including computer vision for autonomous operation of cars or robots, map information, surveillance, infrastructure inspection, and image distribution. Three-dimensional data is obtained through various methods, such as distance sensors such as rangefinders, stereo cameras, or a combination of multiple monocular cameras.

[0003] As one of the methods of expressing three-dimensional data, there is a method called point cloud that expresses the shape of a three-dimensional structure through a group of points in a three-dimensional space. In a point cloud, the position and color of a point group are stored. Although point clouds are expected to become the mainstream method of expressing three-dimensional data, the amount of data in a point group is very large. Therefore, in the accumulation or transmission of three-dimensional data, like two-dimensional dynamic images (for example, MPEG-4AVC or HEVC standardized by MPEG), it is necessary to compress the data volume through encoding.

[0004] In addition, regarding compression of point clouds, some support is provided by a public library (Point Cloud Library) or the like that performs processing related to point clouds.

[0005] In addition, there is known a technique for searching and displaying facilities located around a vehicle using three-dimensional map data (see, for example, Patent Document 1).

[0006] Prior Art Literature

[0007] Patent Literature

[0008] Patent Document 1: International Publication No. 2014 / 020663 Summary of the invention

[0009] Problems to be solved by the invention

[0010] In addition, there is a case where an irreversible compression method is used as the encoding method. In such a case, the decoded point group is not completely consistent with the original point group. Therefore, it is desirable to improve the reproducibility of the decoded point group.

[0011] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve the reproducibility of a decoded point group.

[0012] Means for solving problems

[0013] A decoding method involved in one embodiment of the present invention decodes multiple three-dimensional points, generates a first vertex at a position other than the edge of the first node on the first face of the first node, generates multiple second vertices on multiple edges of the first node, generates a third vertex in the first node based on the multiple second vertices, generates a triangle defined by the first vertex, the second vertex among the multiple second vertices, and the third vertex in the first node, and generates the multiple three-dimensional points on the face of the triangle.

[0014] A coding method involved in one embodiment of the present invention encodes multiple three-dimensional points, generates a first vertex at a position other than the edge of the first node on the first face of the first node, generates multiple second vertices on multiple edges of the first node, generates a third vertex in the first node based on the multiple second vertices, stores information related to the first vertex, the multiple second vertices and the third vertex in a bit stream, the first vertex, the multiple second vertices and the third vertex are generated to generate a triangle defined by the first vertex, the second vertex among the multiple second vertices and the third vertex in the first node, and the multiple three-dimensional points are approximated as the triangle.

[0015] Effects of the Invention

[0016] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve the reproducibility of a decoded point group. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a diagram showing an example of an origin group involved in the embodiment.

[0018] Figure 2 This is a diagram showing an example of a pruned octree according to an embodiment of the present invention.

[0019] Figure 3 This is a diagram showing an example of two-dimensionally displaying leaf nodes according to an embodiment.

[0020] Figure 4 This is a diagram for explaining a method for generating a centroid vertex according to an embodiment.

[0021] Figure 5 This is a diagram for explaining a method for generating a centroid vertex according to an embodiment.

[0022] Figure 6 This is a diagram showing an example of vertex information according to an embodiment.

[0023] Figure 7 It is a diagram showing an example of a triangle aggregate surface involved in the embodiment.

[0024] Figure 8 It is a diagram for explaining the restoration process of the point group involved in the embodiment.

[0025] Fig. 9 It is a diagram showing an example of a point group according to the embodiment.

[0026] Fig.10 This is a diagram showing an example of generating centroid vertices according to an embodiment of the present invention.

[0027] Fig.11 This is a diagram showing an example of generating a triangle (triangle aggregate surface) according to the embodiment.

[0028] Fig.12 It is a diagram showing a cross section of a node according to the embodiment.

[0029] Fig.13 This is a diagram showing an example of generating temporary vertices according to the embodiment.

[0030] Fig.14 This is a diagram showing an example of generating face vertices according to the embodiment.

[0031] Fig.15 This is a diagram showing a syntax example of the GDU involved in the implementation mode.

[0032] Fig.16 This is a flowchart of the encoding process involved in the implementation mode.

[0033] Fig.17 This is a flowchart of the decoding process involved in the implementation mode.

[0034] Fig.18 It is a diagram for explaining Modification 1 of the method for generating face vertices according to the embodiment.

[0035] Fig.19 It is a diagram for explaining Modification 1 of the method for generating face vertices according to the embodiment.

[0036] Fig. 20 It is a diagram for explaining Modification 2 of the method for generating face vertices according to the embodiment.

[0037] Fig.21 It is a diagram for explaining Modification 3 of the method for generating face vertices according to the embodiment.

[0038] Fig. 22 It is a diagram for explaining Modification 3 of the method for generating face vertices according to the embodiment.

[0039] Fig.23 It is a diagram for explaining Modification 4 of the method for generating face vertices according to the embodiment.

[0040] Fig.24 This is a diagram showing an example of a node face existing between two nodes according to an embodiment.

[0041] Fig.25 It is a diagram showing an example of the position of the weighted average of the point group according to the embodiment.

[0042] Fig.26 It is a diagram showing an example of the position of the weighted average of the intersection points according to the embodiment.

[0043] Fig. 27 It is a diagram showing an example of a point group according to the embodiment.

[0044] Fig.28 This is a diagram showing an example of generating centroid vertices according to an embodiment of the present invention.

[0045] Fig.29 It is a diagram showing an example of generating a triangle (triangle aggregate surface) according to the embodiment.

[0046] Fig.30 This is a diagram showing an example of generating face vertices according to the embodiment.

[0047] Fig.31 This is a diagram showing an example of a surface on which connection evaluation and restoration of centroid vertices are performed according to an embodiment.

[0048] Fig.32 This is a flowchart of the encoding process involved in the implementation mode.

[0049] Fig.33 This is a flowchart of the decoding process involved in the implementation mode.

[0050] Fig.34 This is a flowchart of the transmission process of face vertex information involved in the implementation method.

[0051] Fig.35 This is a flowchart of the decoding process of the face vertex information involved in the implementation method.

[0052] Fig.36 This is a diagram showing examples of face vertex candidates according to the embodiment.

[0053] Fig.37 It is a diagram showing an example of arrangement of vertex groups according to an embodiment.

[0054] Fig.38 It is a diagram showing a vector NF involved in the embodiment.

[0055] Fig.39 It is a diagram showing an example of arrangement of vertex groups according to an embodiment.

[0056] Fig.40 It is a diagram showing an example of arrangement of vertex groups according to an embodiment.

[0057] Fig.41 It is a diagram showing an example of arrangement of vertex groups according to an embodiment.

[0058] Fig.42 2 is a diagram showing examples of edge vertices and centroid vertices according to the embodiment.

[0059] Fig.43 It is a diagram showing an example of a correctly generated triangle involved in the embodiment.

[0060] Fig.44 A diagram showing an example of an erroneously generated triangle according to an embodiment.

[0061] Fig.45 The diagram shows examples of edge vertices, centroid vertices, and face vertices according to the embodiment.

[0062] Fig.46 It is a diagram showing an example of how triangles are generated when the simplified sequencing is applied according to the embodiment.

[0063] Fig.47 It is a diagram for explaining the sequencing process involved in the embodiment.

[0064] Fig.48 It is a diagram for explaining the sequencing process involved in the embodiment.

[0065] Fig.49 It is a diagram for explaining the sequencing process involved in the embodiment.

[0066] Fig.50 It is a diagram for explaining the sequencing process involved in the embodiment.

[0067] Fig.51 This is a diagram showing a syntax example of the GDU involved in the implementation mode.

[0068] Fig.52 It is a diagram showing a modified example of the syntax of the GDU involved in the implementation mode.

[0069] Fig.53 It is a diagram showing a modified example of the syntax of the GDU involved in the implementation mode.

[0070] Fig.54 This is a flowchart of the decoding process involved in the implementation mode.

[0071] Fig.55 This is a block diagram of a decoding device according to an embodiment.

[0072] Fig.56 This is a flowchart of the encoding process involved in the implementation mode.

[0073] Fig.57 This is a block diagram of an encoding device involved in an embodiment. DETAILED DESCRIPTION

[0074] A three-dimensional data decoding method according to one embodiment of the present disclosure decodes a plurality of three-dimensional points, generates a first vertex at a position other than an edge of the first node on the first face of the first node, generates a plurality of second vertices on the plurality of edges of the first node, generates a third vertex in the first node based on the plurality of second vertices, generates a triangle defined by the first vertex, the second vertex among the plurality of second vertices, and the third vertex in the first node, and generates the plurality of three-dimensional points on the face of the triangle. Thus, the decoding device can generate the first vertex at a position other than an edge on the face of the node, and generate a triangle using the first vertex. Thus, a triangle other than an edge passing through a node can be generated, so that the reproducibility of the shape of the decoded point group with respect to the origin point group can be improved.

[0075] For example, the first face may be a face common to the first node and the second node adjacent to the first node, and the first vertex may represent a plurality of three-dimensional points located at the periphery of the first face in the first node and the second node. Thus, by using the first vertex to generate a triangle, the reproducibility of the plurality of three-dimensional points located at the periphery of the first face in the decoded point group can be improved.

[0076] For example, the first vertex may be arranged on the first face of the first node based on the position of the third vertex of the first node and the position of the third vertex of the second node adjacent to the first node. Thus, for example, the reproducibility of the shape of the origin group distributed between two second vertices can be improved.

[0077] For example, the first face may be a face common to the first node and the second node adjacent to the first node, the decoding method further generates a plurality of fourth vertices on a plurality of edges of the second node, and generates a fifth vertex in the second node based on the plurality of fourth vertices, and the first vertex is generated based on information indicating that the third vertex is connected to the fifth vertex. Thus, for example, when a ridgeline is formed across a node boundary, the reproducibility of the shape of the ridgeline in the decoded point group can be improved.

[0078] For example, the information may be set for each of the three mutually orthogonal faces of the first node. In this way, it is possible to control whether to generate the first vertex for each face.

[0079] For example, the first vertex may be generated based on information indicating whether a vertex exists at a position other than an edge on the corresponding face, which is provided for each of the three mutually orthogonal faces of the first node. In this way, it is possible to control whether the first vertex is generated on the three faces of the node.

[0080] For example, the decoding method may further receive a bit stream including the information. Thus, the decoding device can use the information included in the bit stream to determine whether to generate the first vertex. Thus, in the decoding device, there is no need to make this determination based on other conditions, etc., so the processing amount of the decoding device can be reduced.

[0081] For example, the information may include position information of the first vertex. Thus, the decoding device can generate the first vertex at an arbitrary position based on the information. Thus, the degree of freedom in generating the position of the first vertex is improved.

[0082] For example, the first vertex may represent the plurality of three-dimensional points in the first node and other plurality of three-dimensional points of other nodes. Thus, through the first vertex, not only the three-dimensional points in the first node but also the three-dimensional points of other nodes can be reproduced. Thus, the reproducibility of the shape of the original point group in the decoded point group can be improved.

[0083] For example, the first vertex may be arranged at a predetermined distance or more from a line connecting two second vertices on the first surface. Thus, the first vertex can be used to reproduce a shape that cannot be reproduced by the second vertex. Thus, the reproducibility of the shape of the decoded point group relative to the original point group can be improved.

[0084] For example, the decoding method may further receive a bit stream including information indicating whether the first vertex is generated on each of the faces of the first node that meet a predetermined condition. This can reduce the amount of information stored in the bit stream.

[0085] For example, the first face may be a face common to the first node and a second node adjacent to the first node, the decoding method may further generate a plurality of fourth vertices on a plurality of edges of the second node, and generate a fifth vertex in the second node based on the plurality of fourth vertices, the predetermined condition may include at least one of a first condition indicating whether the face includes two or three second vertices and a second condition indicating whether the first vector, the second vector, and the third vector are oriented in the same direction, the first vector is a vector from the first centroid of the plurality of second vertices to the third vertex, the second vector is a vector from the second centroid of the plurality of fourth vertices to the fifth vertex, the third vector is a vector from a first line connecting two second vertices of the first face to a temporary first vertex, the temporary first vertex being set at a position where a second line connecting the third vertex and the fifth vertex intersects the first face. Thus, in a case where the possibility of improving reproducibility by generating the first vertex is low, the transmission of information related to the first vertex can be omitted.

[0086] For example, in the generation of the triangle, the inverse tangent of the first vertex and the plurality of second vertices is calculated in a state where the viewpoint is facing the annular distribution including the first vertex and the plurality of second vertices, thereby sequencing the first vertex and the plurality of second vertices, and based on the sequencing result, two vertices are selected from the first vertex and the plurality of second vertices, and the triangle including the selected two vertices and the third vertex is generated. Thus, the decoding device can appropriately generate a triangle when using the first vertex.

[0087] A three-dimensional data encoding method according to one embodiment of the present disclosure encodes a plurality of three-dimensional points, generates a first vertex at a position other than an edge of the first node on the first face of the first node, generates a plurality of second vertices on the plurality of edges of the first node, generates a third vertex in the first node based on the plurality of second vertices, stores information related to the first vertex, the plurality of second vertices, and the third vertex in a bit stream, the first vertex, the plurality of second vertices, and the third vertex are generated to generate a triangle defined by the first vertex, the second vertex among the plurality of second vertices, and the third vertex in the first node, and the plurality of three-dimensional points are approximated to the triangle. Thus, for example, a decoding device can generate a first vertex at a position other than an edge on the face of a node using information included in the bit stream, and generate a triangle using the first vertex. Thus, a triangle other than an edge passing through a node can be generated, so that the reproducibility of the shape of the original point group in the decoded point group can be improved.

[0088] For example, the first face may be a face common to the first node and the second node adjacent to the first node, and the first vertex may be generated based on the plurality of three-dimensional points in the first node and the plurality of three-dimensional points in the second node. Thus, the first vertex is generated based on the three-dimensional points in the first node and the second node, thereby improving the reproducibility of the shape of the original point group in the decoded point group.

[0089] For example, the first vertex may be generated based on a plane or a curved surface in the first node and a plane or a curved surface in the second node. Thus, the first vertex is generated based on the distribution of three-dimensional points in the first node and the second node, thereby improving the reproducibility of the shape of the original point group in the decoded point group.

[0090] For example, the encoding method may further generate a plurality of fourth vertices on a plurality of edges of the second node, generate a fifth vertex in the second node based on the plurality of fourth vertices, and in generating the first vertex, shift the plane or the curved surface in the first node toward the third vertex, shift the plane or the curved surface in the second node toward the fifth vertex, and generate the first vertex based on the shifted plane or the shifted curved surface in the first node and the shifted plane or the shifted curved surface in the second node. Thus, the first vertex is generated based on the distribution of three-dimensional points in the first node and the second node, thereby improving the reproducibility of the shape of the original point group in the decoded point group.

[0091] For example, the offset plane or the offset curved surface in the first node may pass through the third vertex, and the offset plane or the offset curved surface in the second node may pass through the fifth vertex. Thus, the first vertex is generated based on the distribution of the three-dimensional points in the first node and the second node, so that the reproducibility of the shape of the original point group in the decoded point group can be improved.

[0092] In addition, a decoding device involved in one embodiment of the present invention decodes multiple three-dimensional points and is equipped with a processor and a memory. The processor uses the memory to generate a first vertex at a position other than the edge of the first node on the first face of the first node, generates multiple second vertices on multiple edges of the first node, generates a third vertex in the first node based on the multiple second vertices, generates a triangle in the first node defined by the first vertex, the second vertex among the multiple second vertices, and the third vertex, and generates the multiple three-dimensional points on the face of the triangle.

[0093] In addition, a coding device involved in one embodiment of the present invention encodes multiple three-dimensional points, and is equipped with a processor and a memory. The processor uses the memory to generate a first vertex at a position other than the edge of the first node on the first face of the first node, generates multiple second vertices on multiple edges of the first node, generates a third vertex in the first node based on the multiple second vertices, and stores information related to the first vertex, the multiple second vertices and the third vertex in a bit stream. The first vertex, the multiple second vertices and the third vertex are generated to generate a triangle defined by the first vertex, the second vertex among the multiple second vertices, and the third vertex in the first node, and the multiple three-dimensional points are approximated as the triangle.

[0094] In addition, these general or specific methods can also be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, and can also be implemented by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0095] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In addition, the embodiments described below all represent a specific example of the present disclosure. The numerical values, shapes, materials, constituent elements, configuration positions and connection methods of constituent elements, steps, and the order of steps shown in the following embodiments are examples and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims are described as arbitrary constituent elements.

[0096] (Implementation Method)

[0097] The following describes the encoding device (three-dimensional data encoding device) and decoding device (three-dimensional data decoding device) involved in this embodiment. The encoding device encodes three-dimensional data to generate a bit stream. The decoding device decodes the bit stream to generate three-dimensional data.

[0098] Three-dimensional data is, for example, three-dimensional point group data (also referred to as point group data). A point group is a collection of multiple three-dimensional points that represents the three-dimensional shape of an object. Point group data includes position information and attribute information of multiple three-dimensional points. The position information represents the three-dimensional position of each three-dimensional point. In addition, position information is sometimes referred to as geometry information. For example, position information is represented by an orthogonal coordinate system or a polar coordinate system.

[0099] The attribute information indicates, for example, color information, reflectance, transmittance, infrared information, normal vector, or time information, etc. One three-dimensional point may have a single attribute information or may have multiple types of attribute information.

[0100] In addition, the following mainly describes encoding and decoding of position information, but the encoding device can also encode and decode attribute information.

[0101] [Triangle collection method]

[0102] The encoding device according to the present embodiment encodes position information using a triangle set (TriSoup: Triangle-Soup, triangle patch set) method.

[0103] The triangle set method is one of the methods for encoding the position information of the point group data, and is an irreversible compression method. In the triangle set method, the origin group of the processing object is replaced with a set of triangles, and the point group is approximated on the plane. Specifically, the origin group is replaced with the vertex information in the node, and the vertices are connected to each other to generate the triangle group. In addition, the vertex information used to generate the triangle is stored in the bit stream and sent to the decoding device.

[0104] First, the encoding process using the triangle set method will be described. Figure 1 is a graph showing an example of an origin group. Figure 1 As shown, a point group 102 of an object is contained in an object space 101 and includes a plurality of points 103 .

[0105] First, the encoding device performs octree segmentation on the original point group until the specified depth. In the octree segmentation, the object space is divided into 8 nodes (subspaces), and 8 bits of information (occupancy code) are generated to indicate whether each node contains a point group. In addition, the node containing the point group is further divided into 8 nodes, and 8 bits of information are generated to indicate whether each of the 8 nodes contains a point group. This process is repeated until the predetermined level.

[0106] Here, in the conventional octree encoding, for example, the segmentation is repeated until the number of point groups included in the node becomes 1 or below a threshold. On the other hand, in the triangle set method, the octree segmentation is performed until the intermediate level, and the octree segmentation is not performed for the levels below the intermediate level. Such an octree until the intermediate level is called a pruned octree.

[0107] Figure 2 is a diagram showing an example of a pruned octree. Figure 2 As shown, the point group 102 is divided into a plurality of leaf nodes 104 (bottom-level nodes) of the pruned octree.

[0108] Next, the encoding device performs the following processing on each leaf node 104 of the pruned octree. In addition, the leaf node is also referred to as a node below. The encoding device generates a vertex on the edge as a representative point of a point group close to the edge of the node. The vertex is called an edge vertex. For example, an edge vertex is generated for each edge of a plurality of edges (e.g., four parallel edges).

[0109] Figure 3 is an example of two-dimensional display of leaf nodes 104, for example, Figure 1 The xy plane is viewed from the z direction as shown. Figure 3 As shown, an edge vertex 112 is generated on the edge based on the proximity points of the edge among a plurality of points 111 within the leaf node 104 .

[0110] In addition, Figure 3 In the example, the dotted line on the periphery of the leaf node 104 is an edge. In addition, in this example, at a point within a distance of 1 from an edge ( Figure 3 The edge vertex 112 is generated at the weighted average position of the positions of the points included in the range 113 in the range 114. In addition, the unit of distance is, for example, the resolution of the point group, but is not limited to this. In addition, the distance (threshold) is 1 in this example, but it can also be other than 1 and can be variable.

[0111] Next, the encoding device generates a vertex inside the node based on a point group existing in the normal direction of the plane including the plurality of edge vertices. This vertex is referred to as a centroid vertex.

[0112] Figure 4 and Figure 5 is a diagram for explaining the method of generating centroid vertices. First, the encoding device selects, for example, 4 points from the edge vertex group as representative points. Figure 4 In the example shown, edge vertices v1 to v4 are selected. Next, the encoding device calculates an approximate plane 121 passing through the four points. Next, the encoding device calculates a normal line n of the approximate plane 121 and an average coordinate M of the four points. Next, the encoding device calculates one or more points (e.g., Figure 5 The centroid vertex C is generated at the weighted average coordinates of the points included in the range 122 shown.

[0113] Next, the encoding device entropy encodes vertex information as information of edge vertices and centroid vertices, and stores the encoded vertex information in a geometry data unit (GDU) included in the bitstream. In addition, the GDU includes not only vertex information but also information indicating a pruned octree.

[0114] Figure 6 is a diagram showing an example of vertex information. Figure 6 As shown, the point group 102 is converted into vertex information 123 .

[0115] Next, the decoding process of the bit stream generated as described above is described. First, the decoding device decodes the GDU from the bit stream to obtain vertex information. Next, the decoding device connects the vertices to generate a triangle collection surface (TriSoup-Surface) as a triangle group.

[0116] Figure 7 is a diagram showing an example of a triangle set surface. Figure 7 In the example shown, four edge vertices v1 to v4 and a centroid vertex C are generated based on vertex information. In addition, a triangle 131 (triangle collection face) is generated with the centroid vertex C and two edge vertices as vertices. For example, a group of two edge vertices on two adjacent edges is selected, and a triangle 131 is generated with the selected group and the centroid vertex as vertices.

[0117] Figure 8 is a diagram for explaining the restoration process of the point group. By performing the above process on each leaf node, Figure 8 As shown, a three-dimensional model representing the object using a plurality of triangles 131 is generated.

[0118] Next, the decoding device generates points 132 at regular intervals on the surface of the triangle 131 , thereby restoring the position information of the point group 133 .

[0119] [Expression of ridge lines of point group surface]

[0120] In the case of a morphological distribution in which the curved surface of a point group distribution (point group surface) is contained inside a leaf node, the curved surface of the point group does not intersect with the edge (edge), and vertices for representing the curved surface cannot be generated. Therefore, there is a situation where the surface model formed by connecting the vertices cannot reproduce the shape of the original point group.

[0121] Fig. 9 is a diagram showing an example of a point group in which a point group is distributed across nodes 1 and 2 and a ridge line is formed. Fig. 9 As shown, edge vertices 112 are generated based on the distribution of point groups near the edge.

[0122] Fig.10 is a diagram showing an example of generating the centroid vertex in this case. Fig.10 As shown, a centroid vertex 151 is generated in the normal direction of the approximate plane of the edge vertex group.

[0123] Fig.111 is a diagram showing an example of how triangle 131 (triangle collection surface) is generated in this case. Fig.11 As shown in FIG. 1 , multiple vertices (multiple edge vertices and centroid vertices) are connected to generate a triangle 131. At this time, as shown in FIG. Fig.11 As shown in area 152 , the point group near the boundary of the node cannot be reproduced.

[0124] This is because, although the centroid vertex can sample the origin group face, in the current method, it is impossible to generate a vertex between the two centroid vertices of two adjacent nodes. For example, when the ridge line is continuously distributed in the node along one of the x, y, and z axes, the vertex corresponding to the ridge line is not generated because the ridge line does not cross the edge. Therefore, this problem occurs.

[0125] In this embodiment, the encoding device predicts the ridgeline of the point group surface, and when it is determined that two adjacent nodes have the same ridgeline, transmits information for connecting the two centroid vertices of the two adjacent nodes with a line segment to the decoding device. The decoding device uses this information to connect the two centroid vertices, and generates a new vertex (face vertex) at the intersection of the obtained line segment and the shared surface of the two nodes. By using this new vertex, the decoding device can reproduce the ridgeline when generating triangle 131. In addition, a shared surface is a surface shared by two nodes, and is a surface where the two nodes are adjacent.

[0126] By using the above method, the point group surface near the node boundary can be reproduced, so a decoded point group closer to the original point group can be obtained. In addition, in the above description, the point group surface is only used to explain the issues related to the ridgeline, and there is no need to actually find the ridgeline.

[0127] [How to generate face vertices]

[0128] The following describes a method for generating face vertices, which are new vertices for expressing ridge lines. Fig.12 Yes means Fig.10 The diagram is a cross-sectional view of a node 1 and a node 2, and is a diagram showing a cross-sectional view of the yz plane.

[0129] First, the encoding device generates an approximate plane using the positions of the edge vertex group, and moves the approximate plane in parallel so that it includes the centroid vertex, thereby generating a prediction plane. Fig.12 G1 shown is the centroid of the edge vertices of node 1, and G2 is the centroid of the edge vertices of node 2. In addition, C1 is the centroid vertex of node 1, and C2 is the centroid vertex of node 2.

[0130] Next, the encoding device calculates the weighted average of the coordinates of the point group near the intersection line where the prediction plane intersects the shared surface, and generates a temporary vertex at the calculated weighted average position. In addition, the point group near the intersection line is, for example, a plurality of points whose distance from the intersection line is less than a predetermined distance. Fig.13 is a graph showing an example of generating temporary vertices. Fig.13 In the example shown, temporary vertices A1 and A2 are generated at nodes 1 and 2, respectively.

[0131] Specifically, temporary vertices are generated on the shared surface of leaf nodes. In addition, the same processing as the object node is performed on the six adjacent nodes in the six directions of the object node to be processed, and temporary vertices are generated on the shared surface between the adjacent nodes and the object node. For example, Fig.13 In the example shown, node 1 is the target node, node 2 is the adjacent node, a temporary vertex A1 is generated on the shared surface of node 1, and a temporary vertex A2 is generated on the shared surface of node 2.

[0132] Next, the encoding device determines whether to use the temporary vertex as a new vertex. For example, the following two determinations can be used for this determination.

[0133] As a first determination, the encoding device determines whether the distance between the temporary vertex A1 and the temporary vertex A2 on the shared surface is close. In other words, the encoding device determines whether the distance between the temporary vertex A1 and the temporary vertex A2 is smaller than a predetermined first threshold.

[0134] As a second determination, the encoding device determines whether a vector from the centroid vertex C1 of node 1 to the temporary vertex A1, that is, a temporary vector v1, and a vector from the centroid vertex C2 of node 2 to the temporary vertex A2, that is, a temporary vector v2, are substantially parallel. In other words, the encoding device determines whether an angle between the temporary vector v1 and the temporary vector v2 is smaller than a predetermined second threshold value.

[0135] When both the first and second judgments are true, that is, when the temporary vertex A1 is close to the temporary vertex A2 and the temporary vector v1 is substantially parallel to the temporary vector v2, the encoding device determines that a ridge line exists, and adds a face vertex as a new vertex at the intersection of the line segment connecting the centroid vertex C1 of node 1 and the centroid vertex C2 of node 2 and the shared face.

[0136] Furthermore, the encoding device may perform only one of the first determination and the second determination. Alternatively, the encoding device may determine that a ridgeline exists when at least one of the first determination and the second determination is true.

[0137] Fig.14For example, when both the first judgment and the second judgment are true, the encoding device generates a face vertex B at the intersection point where the line segment connecting the centroid vertex C1 and the centroid vertex C2 intersects the shared face.

[0138] [syntax]

[0139] The encoding device performs the above-mentioned determination of the generated surface vertices between the object node and the six adjacent nodes. In addition, the determination is limited to the node faces (candidate faces) intersecting with the prediction plane among the six faces of the object node. In addition, in the decoding device, the candidate faces of the node can also be reduced.

[0140] Fig.15 This is a diagram showing a syntax example of GDU (geometry_data_unit_data) included in a bitstream. GDU includes octree information and geometry_trisoup_data. The octree information is information showing the structure of a pruned octree and is information showing the positions of a plurality of leaf nodes included in the pruned octree.

[0141] geometry_trisoup_data includes the number of edges, edge vertex information, the number of edge vertices, edge vertex position information, the number of candidate faces and face vertex information.

[0142] The number of edges indicates the number of unique edges. In addition, a unique edge is a unique (non-duplicate) edge after excluding edges with duplicate coordinates. Edge vertex information is set for each edge, and edge vertex information [i] indicates whether an edge vertex exists on the i-th edge. For example, a value of 0 indicates that an edge vertex does not exist, and a value of 1 indicates that an edge vertex exists.

[0143] The number of edge vertices indicates the number of vertices on an edge, that is, the number of edge vertices. The edge vertex position information is set for each edge vertex, and the edge vertex position information [i] indicates the position of the i-th edge vertex.

[0144] The candidate face number indicates the number of candidate faces. Face vertex information is set for each candidate face, and face vertex information [i] indicates whether a face vertex exists on the i-th candidate face (whether a face vertex is generated). For example, a value of 0 indicates that a face vertex does not exist, and a value of 1 indicates that a face vertex exists.

[0145] The decoding device restores the information of multiple candidate faces based on the positions of nodes, edge vertices, and centroid vertices. The decoding device sorts the multiple candidate faces in coordinate order, and extracts a unique (non-repeating) candidate face (unique candidate face) after excluding candidate faces with repeated coordinates. Thus, the arrangement order of the multiple candidate faces is uniquely (non-repeating). The decoding device generates face vertices using face vertex information.

[0146] In addition, information indicating the number of all unique faces may be stored in the bitstream instead of the number of candidate faces. Alternatively, the encoding device may not store information indicating the number of all unique faces in the bitstream, and the decoding device may calculate the number of all unique faces based on the number of leaf nodes and the configuration relationship of multiple nodes. Alternatively, the decoding device may calculate a prediction plane and use the prediction plane to calculate the number of candidate faces.

[0147] [Processing Flow]

[0148] The following describes the flow of processing in the encoding device and the decoding device. Fig.16 This is a flowchart of the encoding process performed by the encoding device.

[0149] First, the encoding device generates a pruned octree and generates a plurality of leaf nodes (leaf node groups) (S101). Next, the encoding device performs the following processing of steps S102 to S104 for each of the plurality of leaf nodes of the pruned octree (loop processing 1).

[0150] First, the encoding device determines the edge vertex, the approximate plane and the centroid vertex based on the point group distribution in the node (S102). Next, the encoding device determines the prediction plane based on the approximate plane and the centroid vertex (S103). Next, the encoding device determines the temporary vertex based on the point group distribution near the intersection of the prediction plane and the node surface (S104).

[0151] Thus, the loop process 1 for the target node ends. Next, the encoding device performs the following processes of steps S105 to S111 for each of the plurality of leaf nodes of the pruned octree (loop process 2).

[0152] First, the encoding device determines whether a temporary vertex exists on a shared surface between a target node of a processing target and an adjacent node adjacent to the target node ( S105 ).

[0153] When a temporary vertex exists on the shared surface (S105: Yes), determine whether the distance between the temporary vertex of the object node and the temporary vertex of the adjacent node on the shared surface is below the first threshold, and whether the angle difference between the temporary vector of the object node and the temporary vector of the adjacent node is below the second threshold (S106).

[0154] When the distance is less than or equal to the first threshold value and the angle difference is less than or equal to the second threshold value (S106: Yes), the encoding device sets the face vertex information of the candidate face to a value of 1 (true) (S107).

[0155] On the other hand, when the distance is larger than the first threshold value, or when the angle difference is larger than the second threshold value (S106: No), the encoding device sets the face vertex information of the candidate face to a value of 0 (false) (S108).

[0156] Next, the encoding device adds the face vertex information of the candidate face to the face vertex information list ( S109 ).

[0157] Furthermore, the processing of steps S105 to S109 is performed on each of the six adjacent nodes adjacent to the target node in six directions.

[0158] Next, the encoding device performs entropy encoding on the face vertex information list, and stores the encoded face vertex information list in the GDU ( S110 ).

[0159] After step S110, or when there is no temporary vertex on the shared surface (S105: No), the encoding device performs entropy encoding on the vertex information of the position information of all vertices (edge ​​vertices and centroid vertices) possessed by the object node, and stores the encoded vertex information in the GDU (S111). Thus, loop processing 2 for the object node ends.

[0160] Fig.17 Flowchart of the decoding process performed by the decoding device. First, the decoding device obtains the GDU header and GDU from the bitstream (S121). Next, the decoding device entropy decodes the octree information according to the GDU, and uses the octree information to generate multiple leaf nodes (leaf node groups) of the pruned octree (S122).

[0161] Next, the decoding device performs the following steps S123 to S124 for each of the multiple leaf nodes of the pruned octree (loop processing 1). First, the decoding device obtains vertex information indicating the positions of edge vertices and centroid vertices from the GDU (S123). Specifically, the decoding device performs entropy decoding on the encoded vertex information included in the GDU to obtain the vertex information.

[0162] Next, the decoding device calculates a prediction plane based on the approximate plane and the centroid vertex, and determines a candidate plane (S124). Thus, the loop process 1 for the target node ends.

[0163] Next, the decoding device obtains a plurality of face vertex information (face vertex information list) from the GDU, and generates a candidate face list as a list of all candidate faces (S125). Specifically, the decoding device decodes the encoded face vertex information (face vertex information list) included in the GDU, thereby obtaining a plurality of face vertex information. The decoding device generates a candidate face list using the plurality of face vertex information.

[0164] Next, the decoding device performs the following steps S126 to S128 for each of the leaf nodes of the pruned octree (loop processing 2). First, the decoding device determines whether the object node has a candidate face and the face vertex information of the candidate face is 1 (true) (S126).

[0165] When the object node has a candidate face and the face vertex information of the candidate face is a value of 1 (S126: Yes), the decoding device generates a face vertex at the intersection of the line segment connecting the centroid vertex of the adjacent node that shares the candidate face with the object node and the centroid vertex of the object node, and the shared face (S127). On the other hand, when the object node does not have a candidate face, or when the face vertex information of the candidate face is a value of 0 (false) (S126: No), the decoding device does not generate a face vertex. In addition, when the object node has multiple candidate faces, the processing of steps S126 and S127 is performed for each candidate face.

[0166] Next, the decoding device generates a triangle group using all vertices (edge ​​vertices, centroid vertices, and face vertices) in the target node, and generates points on the surface of each triangle (S128). Thus, loop processing 2 for the target node ends.

[0167] [Variation 1 of the method for generating face vertices]

[0168] As a method for generating face vertices, a method other than the above method may be used. As a first variation of the method for generating face vertices, the following method may be used. Fig.18 and Fig.19 This is a diagram for explaining Modification 1 of the method for generating face vertices.

[0169] The encoding device calculates the centroid vector, which is a vector connected from the approximate plane along the normal direction of the approximate plane to the centroid vertex. The encoding device projects the centroid vector to four parallel faces of the node, and if there is a nearby point group on the projection vector, generates a face vertex at the position of its weighted average.

[0170] For example, Fig.18As shown, the encoding device calculates the centroid vector CV1 connecting the centroid G1 of the edge vertex on the approximate plane to the centroid vertex C1 for node 1. Next, the encoding device projects the centroid vector CV1 to the surface on the negative side of the y direction, the surface on the negative side of the z direction, the surface on the positive side of the y direction, and the surface on the positive side of the z direction of node 1, respectively, thereby generating projection vectors PV11, PV12, PV13, and PV14. The encoding device determines whether there is a point group (nearby point group) near the positions indicated by the projection vectors PV11, PV12, PV13, and PV14 respectively. In addition, the nearby point group is, for example, a plurality of points whose distance from the object position is less than a predetermined distance. When there is a nearby point group, the encoding device generates a face vertex at a weighted average position of the positions of the nearby point groups. Fig.18 In the example, there is a nearby point group relative to the projection vector PV11, such as Fig.19 As shown, the face vertex B1 is generated at a position that is a weighted average of the positions of the nearby point groups.

[0171] Similarly, for node 2, the encoding device calculates the centroid vector CV2 connecting the centroid G2 of the edge vertex on the approximate plane to the centroid vertex C2. Next, the encoding device projects the centroid vector CV2 to the surface on the negative side of the y direction, the surface on the negative side of the z direction, the surface on the positive side of the y direction, and the surface on the positive side of the z direction of node 2, respectively, thereby generating projection vectors PV21, PV22, PV23, and PV24. The encoding device determines whether there is a point group (nearby point group) near the positions indicated by the projection vectors PV21, PV22, PV23, and PV24 respectively, and if there is a nearby point group, generates a face vertex at a position that is a weighted average of the positions of the nearby point groups. Fig.18 In the example, there is a nearby point group relative to the projection vector PV23, such as Fig.19 As shown, the face vertex B2 is generated at the position of the weighted average of the positions of the nearby point groups.

[0172] In the method of this variant 1, there is no need to calculate the predicted plane and the candidate plane, compare the temporary vertices, and compare the temporary vectors, so the amount of processing can be reduced. Furthermore, when generating the face vertices of the object node, the information of the adjacent nodes is not used, so simple processing using only the information of the object node can be achieved.

[0173] [Variation 2 of the method for generating face vertices]

[0174] Next, Modification 2 of the method of generating face vertices will be described. Fig. 20 This is a diagram for explaining Modification 2 of the method for generating face vertices.

[0175] The encoding device generates a face vertex at a weighted average position of a group of points near the intersection of the prediction plane and the face of the node. Fig.12 Next, the encoding device calculates the approximate plane and the predicted plane. Fig. 20 As shown, for node 1, determine the multiple faces between the predicted plane and node 1 (in Fig. 20 In the example shown, whether there is a point group (nearby point group) near the intersection of the four faces (positive side in the x direction, negative side in the x direction, positive side in the y direction, and negative side in the y direction). If there is a nearby point group, the encoding device generates a face vertex at the weighted average position of the positions of the nearby point groups.

[0176] In the method of this modification example 2, there is no need to compare temporary vertices and temporary vectors, thereby reducing the amount of processing. Furthermore, when generating the face vertex of the target node, the information of the adjacent nodes is not used, so simple processing using only the information of the target node can be achieved.

[0177] [Variation 3 of the method for generating face vertices]

[0178] Next, Modification 3 of the method of generating face vertices will be described. Fig.21 and Fig. 22 This is a diagram for explaining Modification 3 of the method for generating face vertices.

[0179] The encoding device calculates an approximate point group surface based on the nearby point group of the prediction plane, calculates an intersection curve between the approximate point group surface and the prediction plane, and generates a surface vertex on a curve vertex of the intersection curve.

[0180] For example, in Fig.21 and Fig. 22 In the example shown, the approximate point group surface and the intersection curve are obtained for each of the nodes 1 and 2. In addition, the surface vertex B1 is generated for the node 1, and the surface vertex B2 is generated for the node 2. Fig.21 and Fig. 22 , only the processing for the shared surface of node 1 and node 2 is recorded, but the same processing is performed on other surfaces.

[0181] Alternatively, a face vertex may be generated when the distance between the vertex of the intersection curve and the edge is greater than a predetermined distance, and no face vertex may be generated when the distance between the vertex and the edge is less than the predetermined distance.

[0182] In the method of this modification example 3, the comparison of the provisional vectors is not necessary, thereby reducing the amount of processing. Furthermore, when generating the face vertices of the target node, the information of the adjacent nodes is not used, so that simple processing using only the information of the target node can be realized.

[0183] [Variation 4 of the method for generating face vertices]

[0184] Next, Modification 4 of the method of generating face vertices will be described. Fig.23 It is a diagram for explaining Modification 4 of the method for generating face vertices.

[0185] When there are a certain number or more of point groups near a line segment connecting two centroid vertices of adjacent nodes, the encoding device generates a surface vertex at an intersection of the line segment and the shared surface.

[0186] For example, in Fig.23 In the example shown, the encoding device temporarily connects the centroid vertex C1 of node 1 and the centroid vertex C2 of node 2 to generate a line segment L0. The encoding device determines whether the number of points existing near the line segment L0 is greater than a threshold value (first determination). That is, the encoding device determines whether the number of points included in the point group (nearby point group) existing in the area within a predetermined distance relative to the line segment L0 is greater than a threshold value. In addition, the encoding device determines whether the distribution of the nearby point group along the line segment L0 is uniform (second determination). For example, it may also be that the encoding device divides the area near the line segment L0 into a plurality of areas, and when the difference in the number of points included in each area is less than a threshold value, it is determined that the distribution of the nearby point group along the line segment L0 is uniform. Alternatively, it may be that the encoding device determines that the distribution of the nearby point group along the line segment L0 is uniform when the number of points included in each area in all areas is greater than a threshold value.

[0187] In addition, when both the first and second determinations are "true", the encoding device determines that the centroid vertices C1 and C2 are connected, and a face vertex is generated at the intersection of the line segment and the shared face. In addition, the encoding device may generate a face vertex when at least one of the first and second determinations is "true". Alternatively, the encoding device may only perform one of the first and second determinations and generate a face vertex when the determination is "true".

[0188] [Other Modifications]

[0189] As a method for generating face vertices, the following method may also be used. For example, as a method for determining a node group for generating vertices on a face of a node, one or more nodes other than the object node may be selected. Alternatively, one or more adjacent nodes that share a face, edge, or node corner point with the object node may be specified.

[0190] As a method for determining the face of the node to generate the vertex in the node group specified in this way, the following method can also be used. It is also possible to select one or more faces of the face of the node existing between two nodes. Here, the "face of the node existing between two nodes" is one or more node faces existing on a straight line obtained by connecting the coordinates of the angles of each of the two nodes. Fig.24is a diagram showing an example of a face of a node existing between two nodes. Fig.24 In the example shown, the two nodes are node 1 and node 4, and the "surfaces of the nodes existing between the two separated nodes" are surface 1, surface 2, and surface 3. In addition, when the two nodes are adjacent, the "surfaces of the nodes existing between the two nodes" are the shared surfaces of the two nodes.

[0191] In addition, as a method for generating face vertices for the face of a node determined in this way, the following method can also be used. In a group of nodes 1 and 2, the encoding device calculates a line segment connecting a feature point of node 1 with a feature point of node 2, and generates face vertices at the intersection of the line segment and the face of the node. The feature point here is one of the vertices (edge ​​vertices or centroid vertices) of the node. Alternatively, the feature point may be one of multiple points of the node. Alternatively, the feature point may be a point representing a feature quantity of a point group within the node. The feature quantity may be, for example, a point obtained by interpolating the position of the point group within the node, or may be a point such as Fig.25 The position of the weighted average of the point group shown. Fig.25 It is a diagram showing an example of the position of the weighted average of the point group.

[0192] Alternatively, as a method for setting a line segment intersecting with a face of a node, the following method may also be used. The encoding device may also define a group for a plurality of feature points of node 1 and a plurality of feature points of node 2 in a group of nodes 1 and 2, and generate a face vertex at a weighted average position of intersection points where a single or multiple line segments connecting the grouped feature points intersect with the face of the node. Fig.26 : is a diagram showing an example of the weighted average position of the intersection points. In this example, the face vertex is generated at the weighted average position of the three intersection points where the shared face of node 1 and node 2 intersects with line segments L1, L2, and L3. In addition, the weight used in the weighted average can also be calculated based on the distance between the feature point and the face of the node, for example.

[0193] In addition, as a method for sending information related to the above-mentioned processing, the following method can also be used. The above-mentioned information for generating face vertices can also be stored in the bit stream when the first mark indicating whether to generate face vertices is valid (for example, value 1), and not stored in the bit stream when the first mark is invalid.

[0194] The first tag is stored in, for example, SPS, GPS, or GDU. SPS (Sequence Parameter Set) is metadata (parameter set) common to multiple frames. GPS (Geometry Parameter Set) is metadata (parameter set) related to encoding of location information. For example, GPS is metadata common to multiple frames.

[0195] Alternatively, the first flag may not be set and the above information may be always stored in the bit stream.

[0196] In addition to the first flag, one or more flags for switching various methods related to the generation of face vertices may be provided separately. For example, the above-mentioned plurality of methods may be switched according to the flag.

[0197] In addition, as the above-mentioned information for generating face vertices, the following information may also be used. This information may also indicate whether face vertices are generated on each face sandwiched between two nodes in the node group of the decoding object. Here, the face sandwiched between two nodes is a shared face of two adjacent nodes sandwiching the face.

[0198] Alternatively, the information may include: identification information (ID) indicating two nodes, and information indicating whether to generate face vertices for all or part of the faces of the nodes on the line segment connecting the two nodes. Fig.24 Node 1 and Node 4 are shown.

[0199] Alternatively, the information may not indicate whether a face vertex is generated on each face among all faces, but may be limited to Fig.12 and Fig.13 The multiple faces of the node intersecting the prediction plane are shown, and for each of the multiple faces, it is indicated whether a face vertex is to be generated on the face.

[0200] Alternatively, the information may indicate whether a face vertex is generated on each of the six faces of each node.

[0201] Alternatively, the information may indicate whether a face vertex is generated on each of the unique plurality of faces after eliminating position duplication.

[0202] Alternatively, the information may indicate whether a face vertex is generated on each face that has no adjacent nodes, or a part of the faces among a plurality of faces of each node.

[0203] In addition, the encoding device may also generate attribute information (such as color or reflectivity) in addition to the position information of the generated face vertices or points generated on the surface of the triangle generated using the face vertices. In addition, the encoding device may also encode the generated attribute information and store the encoded attribute information in the bitstream.

[0204] In addition, in the above-mentioned modified examples 1 to 3 of the method for generating face vertices, the face vertices of the target node can be generated without using information of other nodes. Figure 12 to Figure 14 In the process shown, by directly using the temporary vertex A1 of the node 1 as the face vertex, the face vertex can be generated without using the information of other nodes. The encoding device can also perform such a process of generating face vertices without using the information of other nodes for the node located at the end. The node located at the end is a node having at least one face without an adjacent node.

[0205] In addition, the encoding device can also switch the method of generating the face vertices according to the position of the node. Specifically, the encoding device can also target the nodes that are not located at the end, such as Figure 12 to Figure 14 As in the processing shown or in Modification 4, a method of generating face vertices using information of other nodes is adopted, while a method of not using information of other nodes is adopted for nodes located at the ends.

[0206] In addition, Fig.12 In the illustrated process, the approximation plane is offset so that the prediction plane includes the centroid vertex, but the prediction plane does not necessarily need to include the centroid vertex, and the approximation plane may also be offset toward the centroid vertex.

[0207] In addition, in the above description, an example using a triangle set method is shown, but the method of this embodiment is also effective for other methods other than the triangle set method that use edge vertices of an approximate surface to represent the origin group. For example, the approximate surface may also be a polygonal plane other than a triangle. In addition, the approximate surface may be either a plane or a curved surface.

[0208] In the above description, an example of generating new vertices (face vertices) on the face of a node is shown, but the new vertices do not need to be generated strictly on the face of the node. By generating new vertices inside or outside the node, it is possible to further improve the reproducibility of the origin group.

[0209] In addition, in the above description, the problem related to the ridgeline is solved, but the problem is not limited to the problem related to the ridgeline. Similarly, the solution is not limited to the solution related to the ridgeline. The method of this embodiment is effective for the existing technology that the vertices of the specified approximate surface are limited to the edges.

[0210] [Other examples of expressing ridge lines of point cloud surfaces]

[0211] As described above, in the triangle set method, there is a situation where the shape of the ridge line (ridge line) across adjacent nodes cannot be restored. In contrast, the encoding device generates face vertices on the face that abuts the adjacent nodes, and also restores the point group on the face of the triangle generated based on the centroid vertex, face vertex and edge vertex.

[0212] For example, when there is a curved part of the point group distribution (point group surface) distributed inside the leaf node, the corner of the point group surface does not intersect with the edge, and the vertex cannot be generated at the corner position. Therefore, there is a situation where the surface model formed by connecting the vertices cannot reproduce the shape of the original point group.

[0213] Fig. 27 is a diagram showing an example of a point group in which a point group is distributed across nodes 1 and 2 and a ridge line is formed. Fig. 27 As shown, edge vertices 112 are generated based on the distribution of point groups near the edge.

[0214] Fig.28 is a diagram showing an example of generating the centroid vertex in this case. Fig.28 As shown, a centroid vertex 151 is generated in the normal direction of the approximate plane of the edge vertex group.

[0215] Fig.29 1 is a diagram showing an example of how triangle 131 (triangle collection surface) is generated in this case. Fig.29 As shown in FIG. 1 , multiple vertices (multiple edge vertices and centroid vertices) are connected to generate a triangle 131. At this time, as shown in FIG. Fig.29 As shown, the point group near the node boundary cannot be reproduced.

[0216] This is because, although the centroid vertex can sample the origin group face, in the current method, it is impossible to generate a vertex between the two centroid vertices of two adjacent nodes. For example, when the ridge line is continuously distributed in the node along one of the x, y, and z axes, the vertex corresponding to the ridge line is not generated because the ridge line does not cross the edge. Therefore, this problem occurs.

[0217] In this embodiment, the encoding device predicts the ridgeline of the point group surface, and when it is determined that two adjacent nodes have the same ridgeline, transmits information for connecting the two centroid vertices of the two adjacent nodes with a line segment to the decoding device. This information is, for example, 1-bit information assigned to each face between nodes.

[0218] The decoding device uses this information to connect the centroid vertices and generate new vertices (face vertices) at the intersections of the obtained line segments and the shared faces between the nodes. The decoding device uses this new vertex to reproduce the ridgeline when generating the triangle 131.

[0219] In addition, the coordinate positions of the face vertices are not quantized, so there is no problem of position deviation caused by quantization.

[0220] Fig.30 is a graph showing an example of the generation of face vertices. Fig.30 As shown, the decoding device can reproduce the ridge line by generating face vertices 161 and using the face vertices 161 to generate triangles 131.

[0221] By using the above method, the point group surface near the node boundary can be reproduced, so a decoded point group closer to the original point group can be obtained. In addition, in the above description, the point group surface is only used to explain the issues related to the ridgeline, and there is no need to actually find the ridgeline.

[0222] [Overview of face vertex generation, transfer, and restoration]

[0223] First, the evaluation and restoration of the connectivity between centroid vertices will be described. The encoding device generates a line segment connecting the centroid vertex of the target node with the centroid vertex of the adjacent node for each node, and determines the connectivity between the centroid vertices based on the weights of the point group near the intersection of the line segment and the shared surface between the nodes.

[0224] The encoding device sets a Boolean value (for example, the above-mentioned face vertex information) indicating whether or not a face vertex is generated by connecting two centroid vertices for the corresponding face.

[0225] The Boolean value of each face is transmitted from the encoding device to the decoding device. The decoding device generates a face vertex at a position where a line segment connecting the centroid vertices of nodes on both sides of the face intersects the face for a face having a Boolean value of true.

[0226] Fig.31 is a diagram showing an example of a face among the six faces of a node for which connection evaluation and restoration of a centroid vertex are performed. Fig.31 As shown, for the three faces on the side with larger coordinates of each axis among the six faces of the node, the connection evaluation and restoration of the centroid vertices on both sides of the face are performed.

[0227] Next, the reduction of the amount of data in the bitstream and the limitation of the face information are described in general. In order to reduce the amount of data to be transmitted, the encoding device uses information known to the decoding device to add conditions to the face vertex information (a set of Boolean values) to reduce the amount of data of the face vertex information to be transmitted. In addition, the details of this processing will be described later.

[0228] Next, the ordering of vertex groups within a node is described. In order to generate a triangular collection face, it is necessary to appropriately select two edge vertices, or edge vertices and face vertices. For example, if an edge vertex close to a face vertex is not selected, but an edge vertex far away is selected, a face that is close to the origin group will not be obtained. Furthermore, an approximate face will not be generated. Therefore, for example, in order to generate a face without gaps and holes within a node, it is necessary to use a rotation order to order the edge vertices and face vertex groups based on the centroid vertex. In addition, the details will be described later.

[0229] [Processing Flow]

[0230] Fig.32 201 is a flowchart of the encoding process performed by the encoding device. First, the encoding device performs octree segmentation on the point group to generate a pruned octree, and generates a plurality of leaf nodes (leaf node groups) of the pruned octree. In addition, the encoding device performs arithmetic coding (entropy coding) on ​​the octree information representing the pruned octree, and stores the encoded octree information in the bitstream (S201).

[0231] Next, the encoding device generates edge vertices and centroid vertices for each node among multiple nodes (leaf nodes) according to the distribution of the point group within the node, performs arithmetic encoding (entropy encoding) on ​​the vertex information representing the respective position information, and stores the encoded vertex information in the bit stream (S202).

[0232] Next, the encoding device generates face vertices only for faces that meet the geometric conditions among the multiple faces of each node at the position where a line segment connecting the centroid vertex of the target node and the centroid vertex of the adjacent node intersects the face (S203).

[0233] Next, the encoding device encodes the face vertex information of the face that meets the geometric condition, and stores the encoded face vertex information in the bit stream (S204). Here, the face vertex information is information indicating whether the centroids of the nodes on both sides of the face are connected to generate face vertices.

[0234] Next, the encoding device performs the following steps S205 to S208 (loop processing) for each leaf node of the pruned octree. First, the encoding device sorts the edge vertices and face vertices in the node in reverse time (S205). Next, the encoding device connects the vertex groups (multiple edge vertices, centroid vertices, and face vertices) in the node to generate triangles (triangle collection faces) (S206).

[0235] Next, the encoding device generates a plurality of points on the surface of the triangle (S207). Next, the encoding device uses the coordinate values ​​to uniquely represent the restored point group in the node and appends it to the decoded point group (S208). Here, uniqueness means excluding points with duplicate coordinate values. As described above, the loop processing for the object node ends.

[0236] Fig.33 1 is a flowchart of the decoding process performed by the decoding device. First, the decoding device performs arithmetic decoding (entropy decoding) on ​​the octree information from the bit stream, generates a pruned octree using the octree information, and generates a plurality of leaf nodes (leaf node groups) (S211).

[0237] Next, the decoding device performs arithmetic decoding on vertex information indicating the positions of edge vertices and centroid vertices from the bit stream ( S212 ).

[0238] Next, the decoding device performs arithmetic decoding on the face vertex information only for the faces that meet the geometric conditions among the multiple faces of each leaf node (S213). Next, the decoding device generates face vertices based on the face vertex information (S214).

[0239] Next, the decoding device performs the following steps S215 to S218 (loop processing) for each of the multiple leaf nodes of the pruned octree. First, the decoding device sorts the multiple edge vertices and face vertices in the node in reverse time (S215). Next, the decoding device connects the vertex groups (multiple edge vertices, centroid vertices, and face vertices) in the node to generate triangles (triangle collection faces) (S216).

[0240] Next, the decoding device generates a plurality of points on the surface of the triangle (S217). Next, the decoding device uses the coordinate values ​​to uniquely represent the restored point group in the node and adds it to the decoded point group (S218). Here, uniqueness means excluding points with duplicate coordinate values. As described above, the loop processing for the object node ends.

[0241] Fig.34 It is the transmission process of the face vertex information ( Fig.32 Detailed description of steps S203 and S204 shown in the flowchart).

[0242] The encoding device performs the following steps S221 to S226 (loop processing) for each of the multiple faces of each node. First, the encoding device determines whether the object face of the processing object satisfies the first condition for generating face vertices (S221). In addition, the first condition is a limiting condition based on geometric information for reducing the amount of data in the bit stream, which will be described later. In addition, by setting this condition, faces that cannot generate face vertices can be excluded based on the positional relationship between nodes, edge vertices, and centroid vertices, thereby reducing the amount of transmitted information.

[0243] When the first condition is satisfied (S221: Yes), the encoding device determines whether the target face satisfies the second condition for generating face vertices (S222). In addition, the second condition is an evaluation of the weight of the point group near the position of the face vertex candidate in the connectivity evaluation of the centroid vertex described later. In addition, the details are described later. By setting this condition, it is possible to generate face vertices distributed based on the ridge line shape of the point group on the face.

[0244] When the second condition is satisfied (S222: Yes), the encoding device sets the face vertex information of the target face to "true (vertex exists)" and accumulates the face vertex information for transmission (S223). On the other hand, when the second condition is not satisfied (S222: No), the face vertex information of the target face is set to "false (vertex does not exist)" and accumulates the face vertex information for transmission (S224).

[0245] In addition, when the first condition is not satisfied (S221: No), the encoding device does not generate the surface vertex information of the target surface, regards it as "false", and does not accumulate the surface vertex information for transmission (S225).

[0246] Next, the encoding device generates face vertices on the object face based on the face vertex information (true / false) (S226). That is, if the face vertex information is "true", the encoding device generates face vertices on the object face, and if the face vertex information is "false" (or when it is regarded as "false"), the encoding device does not generate face vertices on the object face. Thus, the loop processing for the object face ends.

[0247] Next, the encoding device encodes the accumulated plurality of face vertex information and stores the encoded face vertex information in the bit stream ( S227 ).

[0248] Fig.35 Is the decoding process of the face vertex information ( Fig.33 Detailed description of steps S213 and S214 is shown in the flowchart.

[0249] The decoding device performs the following steps S231 to S234 (loop processing) on ​​each of the multiple faces of each node. First, the decoding device determines whether the target face satisfies the first condition for generating face vertices (S231). Fig.34 The first condition in step S221 shown is the same.

[0250] When the first condition is satisfied (S231: Yes), the decoding device decodes the face vertex information indicating whether a face vertex is generated on the target face from the bit stream (S232), thereby determining whether a face vertex is generated on the target face ("true" or "false").

[0251] If the first condition is not satisfied ( S231 : No), the decoding device does not decode the surface vertex information of the target surface from the bit stream, but sets the surface vertex information of the target surface to “fake” ( S233 ).

[0252] Next, the decoding device generates face vertices on the object face based on the face vertex information (true / false) (S234). That is, if the face vertex information is "true", the decoding device generates face vertices on the object face, and if the face vertex information is "false", the decoding device does not generate face vertices on the object face. Thus, the loop processing for the object face ends.

[0253] In this way, by determining the first condition (a condition that is determined in advance in the encoding device and the decoding device and cannot be changed) before the second condition of whether to generate a face vertex, it is possible to suppress the transmission of flag information.

[0254] In addition, by combining a predetermined first condition (an unchangeable condition) with a second condition (a changeable condition) that can be flexibly set in the encoding device and notified using a flag, it is possible to achieve both reduction in data volume and flexibility in setting.

[0255] [Evaluation and restoration of connectivity of centroid vertices]

[0256] The encoding device evaluates the weight of the point group on the line segment connecting the centroid vertices, and sets the candidate as the face vertex if the weight of the point group near the face vertex candidate is above a threshold. Here, the candidate is the intersection of the line segment connecting the centroid vertices and the face. In addition, the weight of the point group near the face vertex candidate is the number or density of points contained in the area within a predetermined distance from the face vertex candidate.

[0257] Fig.36 2 is a diagram showing an example of face vertex candidates. In this example, the intersection point where the line segment L1 connecting the centroid vertex C1 of the node 1 and the centroid vertex C2 of the node 2 intersects the shared face F1 is determined as a face vertex candidate.

[0258] The encoding device adds 1-bit information (centroid vertex information) indicating whether to connect the centroid vertices of the nodes on both sides of the face and generate a vertex on the face to the bit stream as transmission information to the decoding device. In addition, the encoding device generates the 1-bit information for each of all faces and stores the generated multiple 1-bit information in the bit stream.

[0259] The decoding device acquires face vertex information in addition to the node position information, edge vertex and centroid vertex position information, and generates face vertices for each face based on the corresponding face vertex information.

[0260] In addition, the encoding device sets the intersection of the line segment of the centroid vertex and the face as the position of the face vertex, but the face vertex can also be generated at a position deviated from the position of the intersection based on the distribution of the point group, for example. In this case, in addition to the above-mentioned 1-bit information, the encoding device also stores the offset of the position of the above-mentioned intersection and the face vertex on the face in the bit stream. The offset is represented by a two-dimensional value, for example. In other words, information representing the position of the face vertex can also be stored in the bit stream. In addition, the information representing the position of the face vertex is not limited to the offset, but can also be coordinate information, or a difference or vector relative to the coordinates of other vertices (edge ​​vertices or centroid vertices).

[0261] In this way, the position of the face vertices maintains the connectivity of the point group face between nodes and better reflects the ridgeline shape of the point group, thereby obtaining a high-quality restored point group.

[0262] [Reduction of the amount of data in the bitstream]

[0263] The encoding device uses the information known by the decoding device to set conditions for all faces, reducing the amount of face vertex information to be transmitted. Specifically, the decoding device knows the position information of edge vertices and centroid vertices, so it uses this information to exclude pairs of centroid vertices that cannot be connected based on geometric relationships.

[0264] For example, the encoding device limits the generation of plane vertex information using AND (logical product) of the following five conditions (a) to (e). The encoding device may use part of these conditions or may further combine other conditions.

[0265] (a) The object node has a centroid vertex (C0). (b) There is a node next to the object node (an adjacent node exists). In addition, an adjacent node may exist in any of the x, y, and z axis directions. (c) The adjacent node has a centroid vertex (C1). If there are few edge vertices, the centroid vertex may not be generated, so this condition is set. (d) The number of edge vertices on the shared surface shared between nodes is 2 or 3. This condition assumes that the point group between nodes exists as a ridge line.

[0266] (e) The face in which face vertices are generated is raised compared to the original face. Here, the face is composed of multiple triangles (triangle collection face). Specifically, when (1) the vector Cvec0 from the centroid G0 of the edge vertex group of the object node to the centroid vertex C0, (2) the vector Cvec1 from the centroid G1 of the edge vertex group of the adjacent node to the centroid vertex C1, and (3) the vector NF when N is the foot of the perpendicular line from the face vertex candidate F to the line segment formed by the two edge vertices on the shared face, none of these three vectors are reversed (the inner product of vector Cvec0 and vector NF and the inner product of vector Cvec1 and vector NF are both positive), the face vertex information of the object face is set as the transmission object.

[0267] Fig.37 This is a diagram showing an example of the arrangement of a vertex group that generates surface vertices. Fig.38 is a graph representing the vector NF. Fig.37 In the example shown, there are two edge vertices on a shared surface and the surface is raised due to the generation of surface vertices, and the surface vertex information is transmitted.

[0268] in addition, Fig.38 The vector u shown is the unit vector of the vector E2E3. x=E2F·u, NF=E2F-xu holds. In addition, when Cvev0·NF>0 and Cvev1·NF>0, the surface is bulging.

[0269] Fig.39 This is a diagram showing an example of arrangement of a vertex group in which two edge vertices exist on a shared surface and the surface is not raised. In this case, the surface vertex information is not transmitted.

[0270] Fig.40 This is a diagram showing an example of arrangement of a vertex group in which two edge vertices exist on a shared surface and the surface is not raised. In this case, the surface vertex information is not transmitted.

[0271] Fig.41 1 is a diagram showing an example of a configuration of a vertex group in which three edge vertices exist on a shared surface and the surface is raised. In this case, the surface vertex information is transmitted. Fig.41In the example shown, a pair of edge vertices (E2, E3) whose midpoint of a line segment formed by two adjacent edge vertices E2, E3, and E4 is closest to the face vertex candidate is selected to determine point N.

[0272] In addition, it is not necessary to make the above determination. Fig.40 The situation shown can also be determined as point group surface bulge.

[0273] In addition, as another condition, the encoding device may determine that face vertices are also generated in the target node when face vertices are continuously generated in a certain number or more in the previously restored nodes connected to the adjacent nodes during the reconstruction process. In addition, the encoding device may determine that the point group face is bulging by actually calculating the volume of the point group face instead of using the sign of the inner product of the above-mentioned vectors, and determining that the face is bulging when the volume increases.

[0274] In addition, the above determination corresponds to Fig.34 Step S221 and Fig.35 The first condition in step S231 shown in FIG. 1 is a condition that the coding device performs the determination. However, the decoding device also performs the same determination.

[0275] [Ordering of vertex groups within a node]

[0276] In order to restore the point group, multiple triangles (triangle collection faces) are generated in the node. At this time, it is necessary to select vertex groups from the end in sequence so as not to miss the generated triangles. Specifically, the decoding device uses the centroid vertex as the center and sorts the edge vertices and face vertices in the rotation order. The decoding device selects 2 points each time based on the set order, and generates triangles with the selected 2 points and the centroid vertex. In this way, triangles can be generated without gaps in the node.

[0277] However, the existing method uses the premise that the sequencing object is only the edge vertices on the node frame, projects the vertex group from the main axis (one of the x, y, and z axes), and realizes the sequencing through simple sorting. In this embodiment, face vertices are generated on the face of the node, so the sequencing object is not limited to the node frame (edge), and the simple sorting is no longer valid.

[0278] Figures 42 to 46 This is a diagram for explaining this topic. Fig.42 is a diagram showing examples of edge vertices and centroid vertices. Fig.43 The figure shows an example of a correctly generated triangle. Fig.44 The figure shows an example of an erroneously generated triangle.

[0279] For example, in Fig.42In the example shown, the edge vertices are sequenced. For example, the decoding device projects the edge vertices in the x-axis direction to sequence. Next, the decoding device Fig.43 As shown in , a triangle is generated by a group consisting of a centroid vertex and two adjacent edge vertices. Fig.44 As shown, if the ordering of edge vertices is wrong, holes are opened in the nodes.

[0280] Fig.45 is a diagram showing examples of edge vertices, centroid vertices, and face vertices. Fig.46 This is a diagram showing an example of how triangles are generated when simple sorting is applied in this case.

[0281] For example, when applying the simplified sort to a vertex group including face vertices, Fig.45 In addition, Fig.45 The numerical values ​​in brackets shown in represent the order set by sequencing. Fig.46 In this example, the order of face vertex F2 should be set to (5), but is set to (7), so a hole occurs.

[0282] In contrast, instead of performing a simple sort, the arc tangent of each vertex is calculated based on the circular distribution formed by the edge vertices and the face vertices, with the viewpoint facing the edge vertices and the face vertices. In addition, a rotation matrix is ​​multiplied to the vertex group in order to make the viewpoint face the circular distribution.

[0283] Figures 47 to 50 It is a diagram for explaining this process. Fig.47 (A) is the normal unit vector of the edge vertex group in the node. (B) is the unit vector in the z-axis direction. (C) is the rotation axis obtained by the outer product of (A) and (B).

[0284] After aligning the origin of vertex coordinates with the centroid vertex, the edge vertices and face vertex group are multiplied by a matrix that makes (A) and (B) rotate in unison, so that the annular arrangement of the vertex group is oriented to the z-axis.

[0285] The rotation amount (cosθ, sinθ) is obtained from the inner product of (A) and (B). The rotation axis (C) is obtained from the outer product of (A) and (B).

[0286] Fig.48 is a diagram showing an example of each vertex in the state facing the z-axis after the above processing. The decoding device calculates the arc tangent (arctan) based on the x and y coordinates of each vertex in the state facing the z-axis, sorts each vertex and adds an order. Fig.49The decoding device takes the centroid vertex as the center, selects 2 points at a time from the vertex group including edge vertices and face vertices in order, and generates a triangle using the selected 2 points and the centroid vertex. Fig.50 Multiple triangles are generated as shown.

[0287] In the above example, the viewpoints in the z-axis direction are arranged to be directly opposite to the annular distribution, but the viewpoints may be set in the x-axis direction, the y-axis direction, or other directions.

[0288] [syntax]

[0289] In connection with reducing the amount of bitstream data, information transmitted from the encoding device to the decoding device in order to restore face vertices is 1-bit information (face vertex information) indicating whether face vertices are generated on each face restricted based on geometry.

[0290] Fig.51 is a diagram showing a syntax example of GDU (geometry_data_unit_data) included in a bitstream. GDU includes octree information and geometry_trisoup_data. geometry_trisoup_data includes the number of edges, edge vertex information, the number of edge vertices, the position information of edge vertices, the number of transmission faces, and face vertex information. In addition, the octree information, the number of edges, the edge vertex information, the number of edge vertices, and the position information of edge vertices are the same as Fig.15 The information described in is the same.

[0291] The number of transmitted faces indicates the total number of face information (face vertex information) transmitted. Face vertex information is set for each face, and face vertex information [i] is a 1-bit information indicating whether a face vertex is generated on the i-th face (whether a face vertex exists). For example, a value of 0 indicates that a face vertex is not generated, and a value of 1 indicates that a face vertex is generated.

[0292] In addition, the transmission face number and face vertex information are included in the bitstream when the face vertex function is valid, but are not included in the bitstream when the face vertex function is not valid. The face vertex function is the above-mentioned process of generating face vertices.

[0293] For example, a flag indicating whether the face vertex function is valid or invalid is set, and whether the face vertex function is valid or invalid is determined based on the flag. The flag is stored in the GPS or GDU header, for example.

[0294] In addition, whether the face vertex function is valid may be set for each node. In this case, multiple flags corresponding to multiple nodes may be stored in the GDU header.

[0295] Fig.52This is a diagram showing a modified example of the syntax of GDU. Fig.52 The syntax shown is relative to Fig.51 The syntax shown is the same as above, except that the GDU includes face vertex group information instead of transmitting face count and face vertex information.

[0296] The face vertex group information indicates whether to generate face vertices on each face among multiple faces. Fig.51 The information of the plurality of face vertex information shown is summarized. In addition, the face vertex group information is included in the bitstream when the face vertex function is valid, and is not included in the bitstream when the face vertex function is not valid.

[0297] exist Fig.52 In the syntax shown, the GDU does not include the number of transmitted faces. Here, the node information, edge vertex position information, and centroid vertex position information for applying geometric constraints to each face are known in the decoding device. Thus, the decoding device can calculate the number of face information actually transmitted (the number of transmitted faces) by applying geometric constraints to all faces segmented according to the neighbor relationship of the nodes.

[0298] Fig.53 This is a diagram showing a modified example of the syntax of GDU. Fig.53 As described above, the syntax shown is an example of a syntax in the case where a face vertex is generated at a position offset from the intersection point of a line segment connecting two centroid vertices and a face. Fig.53 The syntax shown is relative to Fig.51 As shown in the syntax, the GDU also includes an offset (x) and an offset (y).

[0299] The offset (x) and offset (y) are set for each face vertex. The offset (x)[i] represents the offset of the i-th face vertex in the x-axis direction between the above-mentioned intersection and the face vertex. The offset (y)[i] represents the offset of the i-th face vertex in the y-axis direction between the above-mentioned intersection and the face vertex. In other words, the offset (x) and offset (y) represent the two-dimensional offset from the intersection to the face vertex.

[0300] For example, the encoding device may also store the two-dimensional offset in a bitstream after quantizing it. In this case, the bitstream includes a quantization parameter used for quantization. The decoding device uses the quantization parameter to inversely quantize the quantized offset included in the bitstream to restore the original offset.

[0301] [other]

[0302] exist Fig.35In the flowchart shown, the decoding device generates face vertices when the received face vertex information is true, but the face vertices may be generated regardless of the face vertex information. For example, the encoding device may not store the face vertex information in the bitstream, and the decoding device may generate face vertices when the first condition is met, and may not generate face vertices when the first condition is not met.

[0303] In addition, Fig.34 In the flowchart shown, the encoding device determines the value (true or false) of the face vertex information using the first condition and the second condition, but the value of the face vertex information may be determined using only one of the first condition and the second condition.

[0304] In the ordering of the vertex groups within the node, the rotation axis passing through the centroid vertex is obtained based on the outer product of the normal vector of the plane formed by the edge vertex group and an arbitrary coordinate axis, but the method for obtaining the rotation axis for ordering the vertex group (multiple edge vertices and face vertices) is not limited to this. For example, the vertex group can also be projected in an arbitrary axis direction passing through the centroid vertex, and the axis direction can be determined so that the minimum value of the distance between the projection point and the axis is greater than a specified value. In addition, the axis direction can also be determined so that the sum of the squares of the multiple distances is greater than a specified value.

[0305] In addition, in order to reduce the amount of data in the bitstream, a Boolean value (1 bit of face vertex information) is transmitted, but other forms of information may also be transmitted. For example, face vertex information may also represent more than 3 values. For example, a value of 0 may mean "do not generate face vertices", a value of 1 may mean "generate face vertices", and a value of 2 may mean "generate face vertices according to the capabilities of the decoding device". Alternatively, when a Boolean value is used, the decoding device may determine whether to generate face vertices according to the capabilities of the decoding device when the Boolean value is true.

[0306] [Summary]

[0307] As described above, the decoding device (three-dimensional data decoding device) according to the embodiment performs Fig.54 The processing shown. The decoding device decodes multiple three-dimensional points. The decoding device generates a first vertex (e.g., a face vertex) at a position other than the edge of the first node on the first face of the first node (S301), generates multiple second vertices (e.g., edge vertices) on multiple edges of the first node (S302), generates a third vertex (e.g., a centroid vertex) in the first node based on the multiple second vertices (S303), generates a triangle (e.g., a triangle collection face) defined by the first vertex, the second vertex among the multiple second vertices, and the third vertex in the first node (S304), and generates multiple three-dimensional points on the face of the triangle (S305).

[0308] Thus, the decoding device can generate the first vertex at a position other than the edge on the face of the node, and generate a triangle using the first vertex. Thus, a triangle other than the edge passing through the node can be generated, so the reproducibility of the shape of the origin point group in the decoded point group can be improved. Specifically, for a triangle set method in which only edge vertices are set in addition to the centroid vertex, the reproducibility of the shape of the origin point group can be improved.

[0309] For example, the first face is a face common to the first node and the second node adjacent to the first node, and the first vertex represents a plurality of three-dimensional points located around the first face in the first node and the second node. Thus, by using the first vertex to generate a triangle, the reproducibility of the plurality of three-dimensional points located around the first face in the decoded point group can be improved.

[0310] For example, the first vertex is arranged on the first face of the first node based on the position of the third vertex of the first node and the position of the third vertex of the second node adjacent to the first node. This can improve the reproducibility of the shape of the origin group distributed between two second vertices.

[0311] For example, the first face is a face common to the first node and the second node adjacent to the first node, the decoding device further generates a plurality of fourth vertices (e.g., edge vertices) on a plurality of edges of the second node, generates a fifth vertex (e.g., centroid vertex) in the second node based on the plurality of fourth vertices, and the first vertex is located on a line passing through the third vertex and the fifth vertex. Thus, for example, when a ridge line is formed across a node boundary, the reproducibility of the shape of the ridge line in the decoded point group can be improved.

[0312] For example, the first face is a face common to the first node and the second node adjacent to the first node, the decoding device further generates a plurality of fourth vertices (e.g., edge vertices) on a plurality of edges of the second node, generates a fifth vertex (e.g., centroid vertex) in the second node based on the plurality of fourth vertices, and the first vertex is generated based on information indicating that the third vertex is connected to the fifth vertex. Thus, for example, when a ridge line is formed across a node boundary, the reproducibility of the shape of the ridge line in the decoded point group can be improved.

[0313] For example, the information is set for each of the three mutually orthogonal faces of the first node. In this way, it is possible to control whether to generate the first vertex for each face.

[0314] For example, the first vertex is generated based on information indicating whether a vertex exists at a position other than an edge on the corresponding face, which is provided for each of the three mutually orthogonal faces of the first node. In this way, it is possible to control whether the first vertex is generated on the three faces of the node.

[0315] For example, the first vertex is determined based on information indicating that the first vertex exists on at least one face of the first node. In this way, it is possible to control whether the first vertex is generated on each face of the point.

[0316] For example, the decoding method further receives a bit stream including the information. Thus, the decoding device can use the information included in the bit stream to determine whether to generate the first vertex. Thus, in the decoding device, there is no need to make this determination based on other conditions, etc., so the processing amount of the decoding device can be reduced.

[0317] For example, the information includes the position information of the first vertex. Thus, the decoding device can generate the first vertex at an arbitrary position based on the information. Thus, the degree of freedom in generating the position of the first vertex is improved. For example, the information includes vector information representing a vector from the third vertex to the first vertex. For example, the information represents the difference (offset) between the intersection of the line passing through the third vertex and the fifth vertex and the first face relative to the position of the first vertex.

[0318] For example, the first vertex represents multiple three-dimensional points in the first node and multiple other three-dimensional points in other nodes. Thus, through the first vertex, not only the three-dimensional points in the first node can be reproduced, but also the three-dimensional points of other nodes can be reproduced. Thus, the reproducibility of the shape of the original point group in the decoded point group can be improved.

[0319] For example, the first vertex is arranged at a predetermined distance or more from the line connecting the two second vertices on the first surface. Thus, the first vertex can be used to reproduce a shape that cannot be reproduced by the second vertex. Thus, the reproducibility of the shape of the decoded point group with respect to the original point group can be improved.

[0320] For example, the decoding device further receives a bit stream including information indicating whether the first vertex is generated on each of the multiple faces of the first node that satisfy a predetermined condition. This can reduce the amount of information stored in the bit stream.

[0321] For example, the first face is a face common to the first node and the second node adjacent to the first node, and the decoding device further generates multiple fourth vertices (e.g., edge vertices) on multiple edges of the second node, and generates a fifth vertex (e.g., face vertex) in the second node based on the multiple fourth vertices. The predetermined conditions include: a first condition indicating whether the face includes two or three second vertices, and at least one of the second conditions indicating whether the first vector (e.g., vector Cvec0), the second vector (e.g., vector Cvec1), and the third vector (e.g., vector NF) are facing the same direction, the first vector is a vector from the first centroid of the multiple second vertices to the third vertex, the second vector is a vector from the second centroid of the multiple fourth vertices to the fifth vertex, and the third vector is a vector from the first line connecting the two second vertices of the first face to the temporary first vertex, and the temporary first vertex is set at a position where the second line connecting the third vertex and the fifth vertex intersects the first face.

[0322] Thus, when there is a low possibility that reproducibility will be improved by generating the first vertex, the transmission of information related to the first vertex can be omitted.

[0323] For example, in the generation of a triangle, the decoding device calculates the arc tangent of the first vertex and the plurality of second vertices in a state where the viewpoint is directed to the annular distribution including the first vertex and the plurality of second vertices, thereby sequencing the first vertex and the plurality of second vertices, selects two vertices from the first vertex and the plurality of second vertices based on the sequencing result, and generates a triangle including the selected two vertices and a third vertex. Thus, the decoding device can appropriately generate a triangle when using the first vertex.

[0324] Fig.55 It is a module of the decoding device 10. For example, the decoding device 10 includes a processor 11 and a memory 12, and the processor 11 uses the memory 12 to perform the above-mentioned processing.

[0325] In addition, the encoding device (three-dimensional data encoding device) involved in the embodiment performs Fig.56 Processing shown. The encoding device encodes multiple three-dimensional points. The encoding device generates a first vertex (e.g., a face vertex) at a position other than the edge of the first node on the first face of the first node (S311), generates multiple second vertices (e.g., edge vertices) on multiple edges of the first node (S312), generates a third vertex (e.g., a centroid vertex) in the first node based on the multiple second vertices (S313), stores information related to the first vertex, the multiple second vertices, and the third vertex in the bit stream (S314), the first vertex, the multiple second vertices, and the third vertex are generated to generate a triangle defined by the first vertex, the second vertex among the multiple second vertices, and the third vertex in the first node, and the multiple three-dimensional points are approximated as triangles.

[0326] Thus, for example, the decoding device can generate a first vertex at a position other than an edge on the face of the node using information included in the bitstream, and generate a triangle using the first vertex. Thus, a triangle other than an edge passing through the node can be generated, thereby improving the reproducibility of the shape of the original point group in the decoded point group.

[0327] For example, the first face is a face common to the first node and the second node adjacent to the first node, and the first vertex is generated based on a plurality of three-dimensional points in the first node and a plurality of three-dimensional points in the second node. Thus, the first vertex is generated based on the three-dimensional points in the first node and the second node, thereby improving the reproducibility of the shape of the original point group in the decoded point group.

[0328] For example, the first vertex is generated based on the plane or curved surface in the first node and the plane or curved surface in the second node. Thus, the first vertex is generated based on the distribution of three-dimensional points in the first node and the second node, thereby improving the reproducibility of the shape of the original point group in the decoded point group.

[0329] For example, the encoding device further generates multiple fourth vertices (for example, edge vertices) on multiple edges of the second node, generates a fifth vertex (for example, a centroid vertex) in the second node based on the multiple fourth vertices, and in the generation of the first vertex, shifts the plane or curved surface in the first node toward the third vertex, shifts the plane or curved surface in the second node toward the fifth vertex, and generates the first vertex based on the shifted plane or shifted surface in the first node and the shifted plane or shifted surface in the second node.

[0330] Thus, the first vertex is generated based on the distribution of the three-dimensional points in the first node and the second node, so that the reproducibility of the shape of the decoded point group with respect to the original point group can be improved.

[0331] For example, the offset plane or offset curved surface in the first node passes through the third vertex, and the offset plane or offset curved surface in the second node passes through the fifth vertex. Thus, the first vertex is generated based on the distribution of the three-dimensional points in the first node and the second node, so that the reproducibility of the shape of the original point group in the decoded point group can be improved.

[0332] In the above embodiment, the feature of generating vertices on faces other than edges is used in a triangle set method for generating triangles, but it can also be used in other methods of generating polygons other than triangles at edge vertices and decoding point groups on the faces of polygons.

[0333] Fig.57It is a module of the encoding device 20. For example, the encoding device 20 includes a processor 21 and a memory 22, and the processor 21 uses the memory 22 to perform the above-mentioned processing.

[0334] As mentioned above, the encoding device (three-dimensional data encoding device) and the decoding device (three-dimensional data decoding device) etc. which concern on embodiment and modification of this disclosure are demonstrated, but this disclosure is not limited to this embodiment.

[0335] In addition, each processing unit included in the encoding device and decoding device according to the above-mentioned embodiments is typically implemented by an LSI which is an integrated circuit. They may be individually single-chip or partially or entirely single-chip.

[0336] Furthermore, the integrated circuit is not limited to LSI, and can be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI can also be used.

[0337] In addition, in each of the above-mentioned embodiments, each component may be formed by dedicated hardware, or implemented by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

[0338] In addition, the present disclosure can also be implemented as an encoding method (three-dimensional data encoding method) or a decoding method (three-dimensional data decoding method) executed by an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device).

[0339] In addition, the present disclosure may also be implemented as a program that enables a computer, a processor, or a device to perform the above-mentioned encoding method or decoding method. In addition, the present disclosure may also be implemented as a bit stream generated by the above-mentioned encoding method. In addition, the present disclosure may also be implemented as a recording medium having the program or the bit stream recorded thereon. For example, the present disclosure may also be implemented as a non-volatile computer-readable recording medium having the program or the bit stream recorded thereon.

[0340] In addition, the division of the functional modules in the block diagram is an example, and multiple functional modules can also be implemented as one functional module, or one functional module can be divided into multiple modules, or a part of the function can be transferred to other functional modules. In addition, the functions of multiple functional modules with similar functions can also be processed in parallel or in time-sharing by a single hardware or software.

[0341] In addition, the order in which each step in the flowchart is executed is an example order for specifically explaining the present disclosure, and may be an order other than the above. In addition, part of the above steps may be executed simultaneously (in parallel) with other steps.

[0342] The above describes one or more encoding devices and decoding devices involved in the embodiments, and the present disclosure is not limited to the embodiments. As long as it does not deviate from the main purpose of the present disclosure, the embodiments obtained by applying various modifications thought of by those skilled in the art to the present embodiment, and the embodiments constructed by combining the constituent elements in different embodiments can also be included in the scope of one or more embodiments.

[0343] Industrial Applicability

[0344] The present disclosure can be applied to encoding devices and decoding devices.

[0345] Description of reference numerals:

[0346] 10 Decoding device

[0347] 11, 21 Processor

[0348] 12.22 Memory

[0349] 20 Encoding device

[0350] 101 Object Space

[0351] 102, 133 point groups

[0352] 103, 111, 132 points

[0353] 104 leaf nodes

[0354] 112 edge vertices

[0355] 113, 122 Range

[0356] 121 Approximate plane

[0357] 123 Vertex Information

[0358] 131 Triangle

[0359] 151 Centroid Vertex

[0360] 152 Area

[0361] 161 face vertices

Claims

1. A decoding method for decoding multiple three-dimensional points. generating a first vertex on a first face of a first node at a position other than an edge of the first node, generating a plurality of second vertices on a plurality of edges of the first node, generating a third vertex in the first node based on the plurality of second vertices, generating, within the first node, a triangle defined by the first vertex, a second vertex among the plurality of second vertices, and the third vertex; The plurality of three-dimensional points are generated on a face of the triangle.

2. The decoding method according to claim 1, The first surface is a surface common to the first node and a second node adjacent to the first node. The first vertices represent a plurality of three-dimensional points located around the first surface among the first nodes and the second nodes.

3. The decoding method according to claim 1, The first vertex is arranged on the first surface of the first node based on the position of the third vertex of the first node and the position of the third vertex of a second node adjacent to the first node.

4. The decoding method according to claim 1, The first surface is a surface common to the first node and a second node adjacent to the first node. The decoding method further comprises: generating a plurality of fourth vertices on a plurality of edges of the second node, generating a fifth vertex in the second node based on the plurality of fourth vertices, The first vertex is generated based on information indicating that the third vertex is connected to the fifth vertex.

5. The decoding method according to claim 4, The information is set for each of the three mutually orthogonal faces of the first node.

6. The decoding method according to claim 1, The first vertex is generated based on information indicating whether a vertex exists at a position other than an edge on the corresponding face, which is provided for each of the three faces that are orthogonal to each other at the first node.

7. The decoding method according to claim 4 or 6, The decoding method further receives a bit stream comprising the information.

8. The decoding method according to claim 4 or 6, The information includes position information of the first vertex.

9. The decoding method according to claim 1, The first vertex represents the multiple three-dimensional points in the first node and other multiple three-dimensional points in other nodes.

10. The decoding method according to claim 1, The first vertex is arranged at a distance greater than or equal to a predetermined distance from a line connecting two of the second vertices on the first surface.

11. The decoding method according to claim 1, The decoding method further receives a bit stream including information indicating whether the first vertex is generated on each face among a plurality of faces of the first node that satisfies a predetermined condition.

12. The decoding method according to claim 11, The first surface is a surface common to the first node and a second node adjacent to the first node. The decoding method further comprises: generating a plurality of fourth vertices on a plurality of edges of the second node, generating a fifth vertex in the second node based on the plurality of fourth vertices, The predetermined conditions include: at least one of a first condition indicating whether the face includes two or three second vertices and a second condition indicating whether the first vector, the second vector, and the third vector face the same direction, The first vector is a vector from the first centroid of the plurality of second vertices to the third vertex, The second vector is a vector from the second centroid of the plurality of fourth vertices to the fifth vertex, The third vector is a vector from a first line connecting the two second vertices of the first surface to a temporary first vertex, and the temporary first vertex is provided at a position where a second line connecting the third vertex and the fifth vertex intersects the first surface.

13. The decoding method according to claim 1, In said generation of said triangle, In a state where the viewpoint faces the annular distribution including the first vertex and the plurality of second vertices, the arc tangent of the first vertex and the plurality of second vertices is calculated to sort the first vertex and the plurality of second vertices. Based on the sequencing result, two vertices are selected from the first vertex and the plurality of second vertices, and the triangle including the selected two vertices and the third vertex is generated.

14. A coding method for encoding a plurality of three-dimensional points, generating a first vertex on a first face of a first node at a position other than an edge of the first node, generating a plurality of second vertices on a plurality of edges of the first node, generating a third vertex in the first node based on the plurality of second vertices, storing information related to the first vertex, the plurality of second vertices and the third vertex in a bit stream, The first vertex, the plurality of second vertices, and the third vertex are generated to generate a triangle defined by the first vertex, a second vertex among the plurality of second vertices, and the third vertex within the first node, and the plurality of three-dimensional points are approximated as the triangle.

15. The encoding method according to claim 14, The first surface is a surface common to the first node and a second node adjacent to the first node. The first vertex is generated based on the plurality of three-dimensional points in the first node and the plurality of three-dimensional points in the second node.

16. The encoding method according to claim 15, The first vertex is generated based on the plane or curved surface in the first node and the plane or curved surface in the second node.

17. The encoding method according to claim 16, The encoding method further comprises: generating a plurality of fourth vertices on a plurality of edges of the second node, generating a fifth vertex in the second node based on the plurality of fourth vertices, In the generation of the first vertex, offsetting the plane or the curved surface in the first node toward the third vertex, offsetting the plane or the curved surface in the second node toward the fifth vertex, The first vertex is generated based on the offset plane or the offset curved surface in the first node and the offset plane or the offset curved surface in the second node.

18. The encoding method according to claim 17, The offset plane or the offset curved surface in the first node passes through the third vertex, and the offset plane or the offset curved surface in the second node passes through the fifth vertex.

19. A decoding device for decoding a plurality of three-dimensional points, comprising: Processor; and Memory, The processor uses the memory, generating a first vertex on a first face of a first node at a position other than an edge of the first node, generating a plurality of second vertices on a plurality of edges of the first node, generating a third vertex in the first node based on the plurality of second vertices, generating, within the first node, a triangle defined by the first vertex, a second vertex among the plurality of second vertices, and the third vertex; The plurality of three-dimensional points are generated on a face of the triangle.

20. A coding device for encoding a plurality of three-dimensional points, comprising: Processor; and Memory, The processor uses the memory, generating a first vertex on a first face of a first node at a position other than an edge of the first node, generating a plurality of second vertices on a plurality of edges of the first node, generating a third vertex in the first node based on the plurality of second vertices, storing information related to the first vertex, the plurality of second vertices and the third vertex in a bit stream, The first vertex, the plurality of second vertices, and the third vertex are generated to generate a triangle defined by the first vertex, a second vertex among the plurality of second vertices, and the third vertex within the first node, and the plurality of three-dimensional points are approximated as the triangle.

Images