Decoding method, encoding method, decoding device, and encoding device

MYPI2025006189A0Undetermined Publication Date: 2024-10-11PANASONIC INTELLECTUAL PROPERTY CORP OF AMERICA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025006189
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-09-26
Filing Date
2024-03-07
Publication Date
2024-10-11

AI Technical Summary

Technical Problem

Current methods for encoding and decoding three-dimensional data, such as point clouds, require significant data storage and transmission due to the large amount of data involved, necessitating efficient compression techniques to reduce the bitstream size.

Method used

The proposed solution involves a decoding and encoding method that utilizes an octree structure and trisoap method to generate and reduce the bitstream data by selectively providing control information only for surfaces that meet specific conditions, reducing the amount of data required for face vertex generation.

Benefits of technology

This approach effectively reduces the data amount in the bitstream, minimizing processing load and enabling more efficient transmission and storage of three-dimensional data while maintaining data integrity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025006189_DRAWING_001
    Figure PI2025006189_DRAWING_001
  • Figure PI2025006189_DRAWING_002
    Figure PI2025006189_DRAWING_002
  • Figure PI2025006189_DRAWING_003
    Figure PI2025006189_DRAWING_003
Patent Text Reader

Abstract

A decoding method includes: receiving a bitstream including geometry information and first control information (S301), the geometry information indicating nodes constituting an octree structure, the first control information indicating whether a first face of a first node included in the nodes includes a first face vertex provided on the first face except for first edges of the first face; and generating or not generating the first face vertex on the first face according to the first control information (S302). Each of the nodes is a unit for containing three-dimensional points. The first node includes a first centroid vertex and first edge vertices that are used in TriSoup scheme. The first face vertex, the first centroid vertex, and the first edge vertices define a triangle on which three-dimensional points in the first node are disposed. The first control information is provided for a face of the first node that satisfies a predetermined condition. Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

Decoding method, encoding method, decoding device, and encoding device

[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device.

[0002] In the future, devices and services that utilize 3D data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, video distribution, etc. 3D data can be acquired in a variety of ways, including distance sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

[0003] One method of representing three-dimensional data is a representation called a point cloud, which represents the shape of a three-dimensional structure using a group of points in three-dimensional space. In a point cloud, the position and color of the points are stored. Point clouds are expected to become the mainstream method of representing three-dimensional data, but point clouds have a very large amount of data. Therefore, when storing or transmitting three-dimensional data, it is essential to compress the amount of data by encoding, just as with two-dimensional moving images (examples include MPEG-4 AVC or HEVC standardized by MPEG).

[0004] Furthermore, compression of point clouds is partially supported by a public library (Point Cloud Library) that performs processing related to point clouds.

[0005] Furthermore, a technique is known in which three-dimensional map data is used to search for and display facilities located around a vehicle (see, for example, Patent Document 1).

[0006] International Publication No. 2014 / 020663

[0007] In such three-dimensional data encoding and decoding, it is desirable to be able to reduce the amount of data in the generated bitstream.

[0008] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device that can reduce the amount of data in a bitstream.

[0009] A decoding method according to one aspect of the present disclosure includes receiving a bitstream including position information indicating a plurality of nodes constituting an octree structure and first control information indicating whether a first face of a first node included in the plurality of nodes has first face vertices provided other than on a plurality of first edges of the first face, and generating or not generating the first face vertices on the first face according to the first control information, each of the plurality of nodes being a unit including a plurality of three-dimensional points, the first node including a first centroid vertex and a plurality of first edge vertices used in a Trisope method, the first face vertex, the first centroid vertex, and the plurality of first edge vertices defining a triangle in which the three-dimensional points in the first node are arranged, and the first control information is provided for a face of the first node that satisfies a predetermined condition.

[0010] An encoding method according to one aspect of the present disclosure generates position information indicating a plurality of nodes constituting an octree structure, generates first control information indicating whether a first face of a first node included in the plurality of nodes has a first face vertex provided other than a plurality of first edges of the first face, and generates a bit stream including the position information and the first control information, wherein each of the plurality of nodes is a unit including a plurality of three-dimensional points, the first node includes a first centroid vertex and a plurality of first edge vertices used in a Trisope method, the first face vertex, the first centroid vertex, and the plurality of first edge vertices define a triangle in which the three-dimensional points in the first node are arranged, and the first control information is provided for a face of the first node that satisfies a predetermined condition.

[0011] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can reduce the amount of data in a bitstream.

[0012] FIG. 1 is a diagram showing an example of an original point cloud according to the first embodiment. FIG. 2 is a diagram showing an example of a pruned octree according to the first embodiment. FIG. 3 is a diagram showing an example of a two-dimensional display of leaf nodes according to the first embodiment. FIG. 4 is a diagram for explaining a method for generating centroid vertices according to the first embodiment. FIG. 5 is a diagram for explaining a method for generating centroid vertices according to the first embodiment. FIG. 6 is a diagram showing example vertex information according to the first embodiment. FIG. 7 is a diagram showing an example of a Trithorpe surface according to the first embodiment. FIG. 8 is a diagram for explaining a point cloud restoration process according to the first embodiment. FIG. 9 is a diagram showing an example of a point cloud according to the first embodiment. FIG. 10 is a diagram showing an example of centroid vertex generation according to the first embodiment. FIG. 11 is a diagram showing an example of a triangle (Trithorpe surface) generation according to the first embodiment. FIG. 12 is a diagram showing an example of face vertex generation according to the first embodiment. FIG. 13 is a diagram showing an example of a surface for evaluating connectivity of centroid vertices and restoring them according to the first embodiment. FIG. 14 is a flowchart of an encoding process according to the first embodiment. FIG. 15 is a flowchart of a decoding process according to the first embodiment. FIG. 16 is a flowchart of a process for transmitting face vertex information according to the first embodiment. FIG. 17 is a flowchart of a process for decoding face vertex information according to the first embodiment. FIG. 18 is a diagram illustrating an example of face vertex candidates according to the first embodiment. FIG. 19 is a diagram illustrating an example of an arrangement of a vertex group according to the first embodiment. FIG. 20 is a diagram illustrating a vector NF according to the first embodiment. FIG. 21 is a diagram illustrating an example of an arrangement of a vertex group according to the first embodiment. FIG. 22 is a diagram illustrating an example of an arrangement of a vertex group according to the first embodiment. FIG. 23 is a diagram illustrating an example of an arrangement of a vertex group according to the first embodiment. FIG. 24 is a diagram illustrating examples of edge vertices and centroid vertices according to the first embodiment. FIG. 25 is a diagram illustrating an example of a correctly generated triangle according to the first embodiment. FIG. 26 is a diagram illustrating an example of an erroneously generated triangle according to the first embodiment. FIG. 27 is a diagram illustrating examples of edge vertices, centroid vertices, and face vertices according to the first embodiment. FIG. 28 is a diagram illustrating an example of triangle generation when simplified ordering according to the first embodiment is applied. FIG. 29 is a diagram for explaining ordering processing according to the first embodiment.FIG. 30 is a diagram for explaining ordering processing according to the first embodiment. FIG. 31 is a diagram for explaining ordering processing according to the first embodiment. FIG. 32 is a diagram for explaining ordering processing according to the first embodiment. FIG. 33 is a diagram for explaining another method of ordering processing according to the first embodiment. FIG. 34 is a diagram for explaining another method of ordering processing according to the first embodiment. FIG. 35 is a diagram showing an example of syntax of a GDU according to the first embodiment. FIG. 36 is a diagram showing a modified example of syntax of a GDU according to the first embodiment. FIG. 37 is a diagram showing a modified example of syntax of a GDU according to the first embodiment. FIG. 38 is a diagram showing an example of the configuration of a bitstream according to the second embodiment. FIG. 39 is a flowchart of transmission processing of face vertex information according to the second embodiment. FIG. 40 is a flowchart of decoding processing of face vertex information according to the second embodiment. FIG. 41 is a diagram showing an example of syntax of a GDU header and a GDU according to the second embodiment. FIG. 42 is a diagram showing a modified example of syntax of a GDU header and a GDU according to the second embodiment. Fig. 43 is a diagram showing a modified example of a GDU header and GDU syntax according to the second embodiment. Fig. 44 is a flowchart of a decoding process according to the embodiment. Fig. 45 is a block diagram of a decoding device according to the embodiment. Fig. 46 is a flowchart of an encoding process according to the embodiment. Fig. 47 is a block diagram of an encoding device according to the embodiment.

[0013] A three-dimensional data decoding method according to one aspect of the present disclosure includes receiving a bitstream including position information indicating a plurality of nodes constituting an octree structure and first control information indicating whether a first face of a first node included in the plurality of nodes has first face vertices provided other than on a plurality of first edges of the first face, and generating or not generating the first face vertices on the first face according to the first control information, each of the plurality of nodes being a unit including a plurality of three-dimensional points, the first node including a first centroid vertex and a plurality of first edge vertices used in a Trisope method, the first face vertex, the first centroid vertex, and the plurality of first edge vertices defining a triangle in which the three-dimensional points in the first node are arranged, and the first control information is provided for a face of the first node that satisfies a predetermined condition.

[0014] According to this, by providing the first control information for the planes that satisfy the predetermined conditions, the amount of data in the bitstream is reduced compared to when the first control information is provided for all planes, and therefore the amount of processing by the decoding device is reduced.

[0015] For example, the predetermined conditions may include a first condition that the face includes two or three edge vertices, whereby when the possibility of generating a face vertex is low, generation of the first control information is omitted, thereby reducing the data amount of the bitstream.

[0016] For example, the predetermined condition may include a second condition that the first vector, the second vector, and the third vector face in the same direction, the first vector being a vector from a first center of the plurality of first edge vertices to the first centroid vertex, the second vector being a vector from a second center of the plurality of second edge vertices of a second node to a second centroid vertex of the second node, the second node being adjacent to the first node such that the face is tangent to the second node, the third vector being a vector from a first line connecting two edge vertices of the face to a virtual face vertex, and the virtual face vertex being located at a position where a second line connecting the first centroid vertex and the second centroid vertex intersects with the face. This allows generation of the first control information to be omitted when the possibility of generating a face vertex is low, thereby reducing the data amount of the bitstream.

[0017] For example, when the dot product of the first vector and the third vector is positive and the dot product of the second vector and the third vector is positive, it may be determined that the first vector, the second vector, and the third vector are oriented in the same direction. This allows the decoding device to appropriately determine the second condition.

[0018] For example, the predetermined conditions may include a third condition of whether an adjacent node adjacent to the first node has a centroid vertex, whereby when the possibility of generating a face vertex is low, generation of the first control information is omitted, thereby reducing the data amount of the bitstream.

[0019] For example, whether the first surface indicated by the first control information has a first face vertex may be determined based on whether the number or density of points included in an area within a predetermined distance from the intersection of (i) a line segment connecting the first centroid vertex and a second centroid vertex of a second node adjacent to the first node, and (ii) a surface shared by the first node and the second node is equal to or greater than a predetermined threshold. In this way, whether or not to generate a face vertex is appropriately set.

[0020] For example, the bitstream may further include second control information indicating whether the first control information for the multiple faces of the first node is included in the bitstream, whereby the decoding device can switch whether to perform the face vertex generation process using the first control information by referring to the second control information.

[0021] For example, if the second control information indicates that the first control information for the multiple faces of the first node is included in the bitstream, face vertices may be generated or not generated for the face to be processed according to the first control information corresponding to the face to be processed among the multiple faces, and if the second control information indicates that the first control information for the multiple faces of the first node is not included in the bitstream, face vertices may be generated for each of the multiple faces.

[0022] According to this, when face vertices are generated for each of a plurality of faces, the bitstream does not need to include the first control information, thereby reducing the data amount of the bitstream.

[0023] For example, if all of the faces of the first node that satisfy the predetermined condition also satisfy another predetermined condition, the second control information may indicate that the first control information for the faces of the first node is not included in the bitstream. Accordingly, for example, if face vertices are generated for each of all faces that satisfy the predetermined condition, the bitstream does not need to include the first control information. Therefore, the amount of data in the bitstream can be reduced.

[0024] A three-dimensional data encoding method according to one aspect of the present disclosure generates position information indicating a plurality of nodes constituting an octree structure, generates first control information indicating whether a first face of a first node included in the plurality of nodes has a first face vertex provided other than a plurality of first edges of the first face, and generates a bit stream including the position information and the first control information, each of the plurality of nodes being a unit including a plurality of three-dimensional points, the first node including a first centroid vertex and a plurality of first edge vertices used in a Trisope method, the first face vertex, the first centroid vertex, and the plurality of first edge vertices defining a triangle in which the three-dimensional points in the first node are arranged, and the first control information is provided for a face of the first node that satisfies a predetermined condition.

[0025] According to this, by providing the first control information for the planes that satisfy the predetermined conditions, the amount of data in the bitstream is reduced compared to when the first control information is provided for all planes.

[0026] Furthermore, a decoding device according to one aspect of the present disclosure is a decoding device that decodes a plurality of three-dimensional points, the decoding device comprising a processor and a memory, wherein the processor uses the memory to receive a bitstream including position information indicating a plurality of nodes constituting an octree structure and first control information indicating whether a first face of a first node included in the plurality of nodes has first face vertices provided other than on a plurality of first edges of the first face, and generates or does not generate the first face vertices on the first face in accordance with the first control information, each of the plurality of nodes being a unit including a plurality of three-dimensional points, the first node including a first centroid vertex and a plurality of first edge vertices used in a Trisope method, the first face vertex, the first centroid vertex, and the plurality of first edge vertices defining a triangle in which the three-dimensional points in the first node are arranged, and the first control information is provided for a face of the first node that satisfies a predetermined condition.

[0027] Moreover, an encoding device according to one aspect of the present disclosure is an encoding device that encodes a plurality of three-dimensional points, and includes a processor and a memory, wherein the processor uses the memory to generate position information indicating a plurality of nodes that constitute an octree structure, generates first control information indicating whether a first face of a first node included in the plurality of nodes has a first face vertex provided other than a plurality of first edges of the first face, and generates a bit stream including the position information and the first control information, wherein each of the plurality of nodes is a unit including a plurality of three-dimensional points, the first node includes a first centroid vertex and a plurality of first edge vertices used in a Trisope method, the first face vertex, the first centroid vertex, and the plurality of first edge vertices define a triangle in which the three-dimensional points in the first node are arranged, and the first control information is provided for a face of the first node that satisfies a predetermined condition.

[0028] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0029] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0030] (Embodiment 1) Hereinafter, an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device) according to this embodiment will be described. The encoding device generates a bitstream by encoding three-dimensional data. The decoding device generates three-dimensional data by decoding the bitstream.

[0031] The three-dimensional data is, for example, three-dimensional point cloud data (also called point cloud data). A point cloud is a collection of multiple three-dimensional points and represents the three-dimensional shape of an object. The point cloud data includes position information and attribute information of multiple three-dimensional points. The position information indicates the three-dimensional position of each three-dimensional point. Note that the position information may also be called geometry information. For example, the position information is expressed in a Cartesian coordinate system or a polar coordinate system.

[0032] The attribute information indicates, for example, color information, reflectance, transmittance, infrared information, normal vector, time information, etc. One 3D point may have a single piece of attribute information or may have multiple types of attribute information.

[0033] Although the following mainly describes the encoding and decoding of position information, the encoding device may also encode and decode attribute information.

[0034] [TriSoup Method] The encoding device according to this embodiment encodes position information using the TriSoup (Triangle-Soup) method.

[0035] The Trithorpe method is a lossy compression method that encodes the positional information of point cloud data. The Trithorpe method converts the original point cloud to be processed into a set of triangles, and then approximates the point cloud on that plane. Specifically, the original point cloud is converted into vertex information within a node, and the vertices are connected to generate a set of triangles. The vertex information used to generate the triangles is then stored in a bitstream and sent to a decoding device.

[0036] First, the encoding process using the Trithorpe method will be described. Fig. 1 shows an example of an original point cloud. As shown in Fig. 1, a point cloud 102 of an object is contained in an object space 101 and includes a plurality of points 103.

[0037] First, the encoding device divides the source point cloud into an octree to a predetermined depth. In the octree division, the target space is divided into eight nodes (subspaces), and 8-bit information (occupancy code) indicating whether or not a point cloud is included in each node is generated. Furthermore, the node including the point cloud is further divided into eight nodes, and 8-bit information indicating whether or not a point cloud is included in each of the eight nodes is generated. This process is repeated up to a predetermined hierarchy.

[0038] In normal octree coding, division is repeated until the number of points included in a node is reduced to one or a threshold value. On the other hand, in the Trithorpe method, octree division is performed up to an intermediate layer, but not to layers below that layer. Such an octree up to an intermediate layer is called a pruned octree.

[0039] 2 is a diagram showing an example of a pruned octree. As shown in FIG. 2, a point cloud 102 is divided into a plurality of leaf nodes 104 (nodes at the bottom level) of the pruned octree.

[0040] Next, the encoding device performs the following process on each leaf node 104 of the pruned octree. Note that hereinafter, a leaf node may also be simply referred to as a node. The encoding device generates a vertex on the edge as a representative point of a group of points close to the edge of the node. This vertex is called an edge vertex. For example, an edge vertex is generated for each of multiple edges (e.g., four parallel sides).

[0041] 3 is an example of a two-dimensional display of the leaf node 104, for example, a diagram showing the xy plane as viewed from the z direction shown in Fig. 1. As shown in Fig. 3, an edge vertex 112 is generated on the edge based on the nearest points of the edge among the multiple points 111 in the leaf node 104.

[0042] In Fig. 3, the dotted line around the periphery of the leaf node 104 is the edge. In this example, the edge vertex 112 is generated at the weighted average position of the points (points included in the range 113 in Fig. 3) whose distance from the edge is within 1. The unit of distance is, for example, the resolution of the point cloud, but is not limited to this. In this example, this distance (threshold) is 1, but it may be a value other than 1 or may be variable.

[0043] Next, the encoding device generates a vertex inside the node based on a group of points existing in the normal direction of the plane containing the edge vertices. This vertex is called a centroid vertex.

[0044] 4 and 5 are diagrams illustrating a method for generating a centroid vertex. First, the encoding device selects, for example, four points as representative points from a group of edge vertices. In the example shown in FIG. 4, edge vertices v1 to v4 are selected. Next, the encoding device calculates an approximate plane 121 that passes through the four points. Next, the encoding device calculates a normal n to the approximate plane 121 and an average coordinate M of the four points. Next, the encoding device generates a centroid vertex C at the weighted average coordinate of one or more points (for example, points included in the range 122 shown in FIG. 5) that are close to a half-line extending from the average coordinate M in the direction of the normal n.

[0045] Next, the encoding device entropy-encodes the vertex information, which is information on the edge vertices and centroid vertices, and stores the encoded vertex information in a Geometry Data Unit (GDU) included in the bitstream. Note that the GDU includes information indicating the pruned octree in addition to the vertex information.

[0046] 6 is a diagram showing an example of vertex information. Through the above processing, the point group 102 is converted into vertex information 123 as shown in FIG.

[0047] Next, the decoding process of the bitstream generated above will be described. First, the decoding device decodes the GDU from the bitstream to obtain vertex information. Next, the decoding device connects the vertices to generate a TriSoup-Surface, which is a group of triangles.

[0048] 7 is a diagram showing an example of a Trithorpe surface. In the example shown in FIG. 7, four edge vertices v1 to v4 and a centroid vertex C are generated based on the vertex information. In addition, a triangle 131 (Trithorpe surface) is generated whose vertices are the centroid vertex C and two edge vertices. For example, pairs of two edge vertices on two adjacent edges are selected, and a triangle 131 is generated whose vertices are the selected pairs and the centroid vertex.

[0049] 8 is a diagram for explaining the point cloud restoration process. By performing the above process for each leaf node, a three-dimensional model is generated in which the target object is represented by a plurality of triangles 131, as shown in FIG.

[0050] Next, the decoding device generates points 132 at regular intervals on the surface of the triangle 131 to restore the position information of the point group 133 .

[0051] [Example of Representation of Ridge Lines of Point Cloud Surfaces] In some cases, the Trithorpe method cannot reconstruct the shape of ridge lines (edge ​​lines) that cross adjacent nodes. In response to this, the encoding device generates face vertices on the faces that abut adjacent nodes, and reconstructs point clouds on the triangular faces generated from the centroid vertices, face vertices, and edge vertices.

[0052] For example, if there are bends in the point cloud distribution (point cloud surface) inside a leaf node, the corners and edges of the point cloud surface do not intersect, and no vertices are created at the corner positions, so the surface model created by connecting the vertices may not be able to reproduce the shape of the original point cloud.

[0053] 9 is a diagram showing an example of a point cloud in which a ridge line is formed by distributing the point cloud across nodes 1 and 2. As shown in FIG. 9, an edge vertex 112 is generated based on the point cloud distribution near the edge.

[0054] 10 is a diagram showing an example of generating a centroid vertex in this case. As shown in Fig. 10, a centroid vertex 151 is generated in the normal direction of the approximation plane of the edge vertex group.

[0055] Fig. 11 shows an example of the generation of a triangle 131 (Trithorpe surface) in this case. As shown in Fig. 11, the triangle 131 is generated by connecting multiple vertices (multiple edge vertices and centroid vertices). In this case, as shown in Fig. 11, the point cloud near the node boundary cannot be reproduced.

[0056] This is because the centroid vertices can sample the original point cloud surface, but the current method cannot create a vertex between two centroid vertices of two adjacent nodes. For example, if a ridge line continues to be distributed within a node along one of the x, y, or z axes, a vertex corresponding to the ridge line will not be generated because the ridge line does not cross an edge. This causes this problem.

[0057] In this embodiment, the encoding device predicts the ridge line of the point cloud surface, and if it determines that two adjacent nodes have the same ridge line, it transmits information for connecting the two centroid vertices of the two adjacent nodes with a line segment to the decoding device. This information is, for example, one bit of information assigned to each face between the nodes.

[0058] The decoder uses this information to connect the centroid vertices and generate new vertices (face vertices) at the intersections of the resulting line segments and the shared faces between the nodes. The decoder can use these new vertices to reconstruct the ridge lines when generating triangles 131.

[0059] Furthermore, since the coordinate positions of the face vertices are not quantized, there is no problem of positional deviation due to quantization.

[0060] 12 is a diagram showing an example of generating face vertices. As shown in Fig. 12, the decoding device generates face vertices 161 and generates triangles 131 using the face vertices 161, thereby reproducing the ridge line.

[0061] The above method can reproduce the point cloud surface near the node boundary, resulting in a decoded point cloud that is closer to the original point cloud. Note that in the above explanation, the point cloud surface is used only to explain the issues related to ridge lines, and the ridge lines do not need to be actually obtained.

[0062] [Overview of Generation, Transmission, and Restoration of Face Vertices] First, we will explain the evaluation and restoration of connectivity between centroid vertices. For each node, the encoding device generates a line segment connecting the centroid vertex of the target node to the centroid vertex of the adjacent node, and determines the connectivity between the centroid vertices from the weight of the point cloud near the intersection of the line segment and the shared face between the nodes.

[0063] The encoding device sets a Boolean value (for example, the face vertex information described above) indicating whether or not to connect two centroid vertices for the corresponding face to generate a face vertex.

[0064] The Boolean value of each face is transmitted from the encoding device to the decoding device, and for faces with a Boolean value = true, the decoding device generates a face vertex at the position where the face intersects with the line segment connecting the centroid vertices of the nodes on both sides of the face.

[0065] 13 is a diagram showing an example of faces, among the six faces of a node, for which connectivity evaluation and restoration of centroid vertices are performed. For example, as shown in FIG. 13, the encoding device and decoding device perform connectivity evaluation and restoration of centroid vertices on both sides of the faces for the three faces with larger coordinates on each axis among the six faces of the node.

[0066] Next, we will explain the outline of reducing the data amount of the bitstream and limiting the face information. To suppress the amount of data to be transmitted, the encoding device applies conditions to the face vertex information (a set of Boolean values) using information already known to the decoding device, thereby reducing the data amount of the face vertex information to be transmitted. Details of this process will be described later.

[0067] Next, we will explain the ordering of vertices within a node. To generate a Trithorpe surface, it is necessary to appropriately select two edge vertices, or an edge vertex and a face vertex. For example, if an edge vertex close to a face vertex is not selected, and an edge vertex far from the face vertex is selected, the surface will not approximate the original point group. Furthermore, an approximate face will not be generated. Therefore, in order to create a surface without gaps or holes within the node, for example, it is necessary to order the edge vertices and face vertices using a rotation order based on the centroid vertex. This will be described in detail later.

[0068] [Processing Flow] Fig. 14 is a flowchart of the encoding process by the encoding device. First, the encoding device generates a pruned octree by dividing a point cloud into octrees, and generates multiple leaf nodes (leaf node group) of the pruned octree. The encoding device then arithmetically encodes (entropy encodes) octree information indicating the pruned octree, and stores the encoded octree information in a bitstream (S201).

[0069] Next, for each of the multiple nodes (leaf nodes), the encoding device generates edge vertices and centroid vertices from the distribution of point clouds within the node, arithmetically encodes (entropy encodes) the vertex information indicating each node's position information, and stores the encoded vertex information in a bitstream (S202).

[0070] Next, the encoding device generates a face vertex at the position where the line segment connecting the centroid vertex of the target node and the centroid vertex of the adjacent node intersects with the face, only for the faces among the multiple faces of each node that meet the geometric conditions (S203).

[0071] Next, the encoding device encodes the face vertex information of the faces that meet the geometry conditions and stores the encoded face vertex information in the bitstream (S204). Here, the face vertex information is information that indicates whether or not to generate a face vertex by connecting the centroids of the nodes on both sides of the face.

[0072] Next, the encoding device performs the following processing (loop processing) of steps S205 to S208 for each of the leaf nodes of the pruned octree. First, the encoding device orders the edge vertices and face vertices in the node counterclockwise (S205). Next, the encoding device connects the vertices in the node (the edge vertices, centroid vertices, and face vertices) to generate a triangle (a trithorpe surface) (S206).

[0073] Next, the encoding device generates multiple points on the surface of the triangle (S207). Next, the encoding device makes the restored point group within the node unique using coordinate values ​​and adds them to the decoded point group (S208). Here, making them unique means excluding points with duplicate coordinate values. This completes the loop processing for the target node.

[0074] 15 is a flowchart of the decoding process performed by the decoding device. First, the decoding device arithmetically decodes (entropy decodes) octet information from the bitstream, generates a pruned octet using the octet information, and generates a plurality of leaf nodes (leaf node group) (S211).

[0075] Next, the decoding device arithmetically decodes vertex information indicating the positions of edge vertices and centroid vertices from the bitstream (S212).

[0076] Next, the decoding device arithmetically decodes face vertex information only for faces that meet the geometry conditions among the multiple faces of each leaf node (S213). Next, the decoding device generates face vertices based on the face vertex information (S214).

[0077] Next, the decoding device performs the following processing (loop processing) of steps S215 to S218 for each of the leaf nodes of the pruned octree. First, the decoding device orders the edge vertices and face vertices in the node counterclockwise (S215). Next, the decoding device generates a triangle (a trithorpe surface) by connecting the vertices in the node (the edge vertices, the centroid vertex, and the face vertices) (S216).

[0078] Next, the decoding device generates multiple points on the surface of the triangle (S217). Next, the decoding device makes the restored point group within the node unique using coordinate values ​​and adds them to the decoded point group (S218). Here, making them unique means excluding points with duplicate coordinate values. This completes the loop processing for the target node.

[0079] FIG. 16 is a flowchart showing the face vertex information transmission process (details of steps S203 and S204 shown in FIG. 14).

[0080] The encoding device performs the following processing (loop processing) of steps S221 to S226 for each of the multiple faces of each node. First, the encoding device determines whether the target face to be processed satisfies a first condition for generating face vertices (S221). Note that the first condition is a restrictive condition based on geometry information for reducing the data volume of the bitstream described below, and will be described in detail later. Furthermore, by setting this condition, it is possible to exclude faces for which face vertices cannot be generated based on the positional relationships between the node, edge vertices, and centroid vertices, thereby reducing the amount of information transmitted.

[0081] If the first condition is satisfied (Yes in S221), the encoding device determines whether the target surface satisfies a second condition for generating face vertices (S222). The second condition refers to an evaluation of the weight of points near the positions of face vertex candidates in the connectivity evaluation of centroid vertices, which will be described later. This will be described in detail later. By setting this condition, face vertices can be generated based on the distribution of the ridge line shape of the points on the surface.

[0082] If the second condition is met (Yes in S222), the encoding device sets the face vertex information of the target face to "true (vertices present)" and stores the face vertex information for transmission (S223). On the other hand, if the second condition is not met (No in S222), the encoding device sets the face vertex information of the target face to "false (vertices absent)" and stores the face vertex information for transmission (S224).

[0083] If the first condition is not met (No in S221), the encoding device does not generate face vertex information for the target surface, treats it as "false", and does not store the face vertex information for transmission (S225).

[0084] Next, the encoding device generates face vertices on the target surface based on the face vertex information (true / false) (S226). That is, if the face vertex information is "true," the encoding device generates face vertices on the target surface, and if the face vertex information is "false" (or is considered to be "false"), the encoding device does not generate face vertices on the target surface. This completes the loop processing for the target surface.

[0085] Next, the encoding device encodes the accumulated pieces of face vertex information and stores the encoded face vertex information in a bitstream (S227).

[0086] FIG. 17 is a flowchart showing the process of decoding face vertex information (details of steps S213 and S214 shown in FIG. 15).

[0087] The decoding device performs the following processing (loop processing) of steps S231 to S234 for each of the multiple faces of each node. First, the decoding device determines whether the target face satisfies a first condition for generating face vertices (S231). Note that the first condition is the same as the first condition in step S221 shown in FIG. 16.

[0088] If the first condition is satisfied (Yes in S231), the decoding device decodes face vertex information indicating whether or not to generate face vertices on the target surface from the bitstream (S232), thereby determining whether or not to generate face vertices on the target surface ("true" or "false").

[0089] Furthermore, if the first condition is not satisfied (No in S231), the decoding device does not decode the face vertex information of the target surface from the bitstream, and sets the face vertex information of the target surface to "false" (S233).

[0090] Next, the decoding device generates face vertices on the target surface based on the face vertex information (true / false) (S234). That is, if the face vertex information is "true", the decoding device generates face vertices on the target surface, and if the face vertex information is "false", the decoding device does not generate face vertices on the target surface. This completes the loop processing for the target surface.

[0091] In this way, by setting the first condition (a condition that is predetermined by the encoding device and the decoding device and cannot be changed) before the second condition of whether or not to generate a face vertex, the transmission of flag information can be suppressed.

[0092] Furthermore, by combining a predetermined first condition (a condition that cannot be changed) with a second condition (a condition that can be changed) that can be flexibly set in the encoding device and notified by a flag, it is possible to achieve both a reduction in the amount of data and flexibility in the settings.

[0093] [Evaluating and Restoring Connectivity of Centroid Vertices] The encoding device evaluates the weight of points on the line segments connecting the centroid vertices, and if the weight of points near a face vertex candidate is equal to or greater than a threshold, the candidate is set as a face vertex. Here, a candidate is the intersection of the line segment connecting the centroid vertices and the face. The weight of points near a face vertex candidate is the number or density of points included in an area within a predetermined distance from the face vertex candidate.

[0094] 18 is a diagram showing an example of face vertex candidates. In this example, the intersection of a line segment L1 connecting the centroid vertex C1 of node 1 and the centroid vertex C2 of node 2 with the shared face F1 is determined as the face vertex candidate.

[0095] The encoding device adds one bit of information (centroid vertex information) to the bit stream as transmission information to the decoding device, which indicates whether or not a vertex is to be generated on the surface by connecting the centroid vertices of the nodes on both sides of the surface. The encoding device also generates this one bit of information for each of the entire surfaces and stores the generated multiple pieces of one bit information in the bit stream.

[0096] The decoder obtains face vertex information in addition to node position information, edge vertex and centroid vertex position information, and generates face vertices for each face based on the corresponding face vertex information.

[0097] Here, the encoding device sets the intersection of the centroid vertex's line segment and the surface as the position of the face vertex. However, the face vertex may be generated at a position shifted from the intersection, for example, based on the distribution of the point cloud. In this case, in addition to the one-bit information, the encoding device stores an offset amount on the surface between the intersection and the position of the face vertex in the bitstream. This offset amount is expressed, for example, as a two-dimensional value. In other words, information indicating the position of the face vertex may be stored in the bitstream. Note that the information indicating the position of the face vertex is not limited to the offset amount, and may be coordinate information, or a coordinate difference or vector from another vertex (edge ​​vertex or centroid vertex).

[0098] This method allows the positions of the face vertices to better reflect the shape of the ridge lines of the point cloud while maintaining the connectivity of the point cloud surface between nodes, resulting in a high-quality restored point cloud.

[0099] [Reducing the amount of data in the bitstream] The encoding device sets conditions for all faces using information known to the decoding device, thereby reducing the amount of face vertex information to be transmitted. Specifically, since the decoding device knows the position information of edge vertices and centroid vertices, it uses this information to exclude pairs of centroid vertices that cannot be connected based on their geometric relationships.

[0100] For example, the encoding device limits the generation of face vertex information by ANDing the following five conditions (a) to (e): Note that the encoding device may use some of these conditions, or may further combine other conditions.

[0101] (a) The target node has a centroid vertex (C0). (b) There is a node next to the target node (there is an adjacent node). Note that it is also possible for there to be an adjacent node in either the x, y, or z axis direction. (c) The adjacent node has a centroid vertex (C1). This condition is set because a centroid vertex may not be generated if there are few edge vertices. (d) The number of edge vertices on the shared surface shared between nodes is 2 or 3. This condition assumes that a point cloud exists in the form of a ridge line between the nodes.

[0102] (e) The surface resulting from the generation of face vertices is larger than the original surface. Here, the surface is a surface composed of multiple triangles (triangle surface). Specifically, if the following three vectors are not inverted (the inner product of vector Cvec0 and vector NF and the inner product of vector Cvec1 and vector NF are both positive): (1) vector Cvec0 from the center of gravity G0 of the edge vertices of the target node to centroid vertex C0; (2) vector Cvec1 from the center of gravity G1 of the edge vertices of the adjacent node to centroid vertex C1; and (3) vector NF when the foot of the perpendicular line drawn from face vertex candidate F to the line segment formed by the two edge vertices on the shared surface is N), the face vertex information of the target surface is set as a transmission target.

[0103] Fig. 19 is a diagram showing an example of the arrangement of vertices from which face vertices are generated. Fig. 20 is a diagram showing a vector NF. In the example shown in Fig. 19, there are two edge vertices on a shared surface, and the surface expands as face vertices are generated, and face vertex information is transmitted.

[0104] 20 is a unit vector of vector E2E3. x=E2F·u, NF=E2F−xu. If Cvev0·NF>0 and Cvev1·NF>0, the surface expands.

[0105] 21 shows an example of the arrangement of vertices where two edge vertices exist on a shared surface and the surface does not bulge. In this case, face vertex information is not transmitted.

[0106] 22 shows an example of the arrangement of vertices where two edge vertices exist on a shared surface and the surface does not bulge. In this case, face vertex information is not transmitted.

[0107] 23 shows an example of the arrangement of a group of vertices where three edge vertices exist on a shared surface and the surface bulges. In this case, face vertex information is transmitted. In the example shown in FIG. 23, the pair of edge vertices (E2, E3) that is closest to the midpoint of the line segment formed by the two adjacent edge vertices E2, E3, and E4 is selected, and point N is determined.

[0108] It should be noted that the above determination does not necessarily have to be made. For example, in the case shown in Fig. 22, it may be determined that the point cloud surface bulges.

[0109] As another condition, the encoding device may determine that face vertices are to be generated in the target node if a certain number or more of face vertices have been generated consecutively in a preceding restored node connected to an adjacent node during the reconstruction process.In addition, the encoding device may determine that the point cloud surface is expanding by actually calculating the volume of the point cloud surface, rather than by using the sign of the dot product of the above-mentioned vectors, and determining that the surface is expanding if the volume increases.

[0110] The above determination corresponds to the first condition in step S221 shown in Fig. 16 and step S231 shown in Fig. 17. Also, although an example in which the encoding device makes the determination has been described here, a similar determination is also made in the decoding device.

[0111] [Ordering vertices within a node] Multiple triangles (trisorp surfaces) are generated within a node for point cloud reconstruction. In this process, the vertices must be selected in order from the edge to avoid missing any triangles. Specifically, the decoding device orders the edge vertices and face vertices in a rotational order, centered on the centroid vertex. The decoding device selects two points at a time based on the set order, and generates a triangle using the selected two points and the centroid vertex. This allows triangles to be generated without gaps within the node.

[0112] However, existing methods use the premise that the ordering targets are only edge vertices on the node frame, project the vertices from the major axis (either x, y, or z axis), and achieve ordering through simple sorting. In this embodiment, face vertices are generated on the face of the node, so the ordering targets are no longer limited to the node frame (edge), and simple sorting no longer works.

[0113] 24 to 28 are diagrams for explaining this problem. Fig. 24 is a diagram showing examples of edge vertices and centroid vertices. Fig. 25 is a diagram showing an example of a correctly generated triangle. Fig. 26 is a diagram showing an example of an erroneously generated triangle.

[0114] For example, in the example shown in Figure 24, edge vertices are ordered. For example, the decoding device projects the edge vertices in the x-axis direction and performs the ordering. Next, the decoding device generates a triangle using a centroid vertex and a pair of two adjacent edge vertices, as shown in Figure 25. On the other hand, as shown in Figure 26, if the edge vertices are ordered incorrectly, a hole will appear in the node.

[0115] Fig. 27 is a diagram showing examples of edge vertices, centroid vertices, and face vertices, and Fig. 28 is a diagram showing an example of triangle generation when simple sorting is applied in this case.

[0116] For example, when simple sorting is applied to a group of vertices including face vertices, the ordering shown in Figure 27 is performed. Note that the numbers in parentheses in Figure 27 indicate the order set by the ordering. In this case, a hole is created as shown in Figure 28. In this example, the order of face vertex F2 should be set to (5), but it has been set to (7), causing a hole.

[0117] Instead of simple sorting, the arctangent of each vertex is calculated after the viewpoint is aligned with the circular distribution of edge vertices and face vertices. In addition, to orient the viewpoint with the circular distribution, a rotation matrix is ​​multiplied on the vertices.

[0118] 29 to 32 are diagrams for explaining this processing. In Fig. 29, (A) is the normal unit vector of the edge vertices within the node. (B) is the unit vector in the z-axis direction. (C) is the rotation axis calculated by the cross product of (A) and (B).

[0119] By aligning the origin of the vertex coordinates with the centroid vertex and then multiplying the edge vertices and face vertices by a matrix that rotates (A) to (B), the circular arrangement of the vertices faces the z-axis.

[0120] The amount of rotation (cosθ, sinθ) is found from the dot product of (A) and (B). The axis of rotation (C) is found from the cross product of (A) and (B).

[0121] FIG. 30 is a diagram showing an example of each vertex facing the z-axis after the above processing. The decoding device calculates the arctangent from the x and y coordinates of each vertex facing the z-axis, and sorts and orders the vertices. This results in the ordering shown in FIG. 31. The decoding device selects two points from a group of vertices including edge vertices and face vertices in order, centered on the centroid vertex, and generates a triangle using the selected two points and the centroid vertex. This generates multiple triangles as shown in FIG. 32.

[0122] In the above example, the circular distribution is made to face the viewpoint in the z-axis direction, but the viewpoint may be set in the x-axis direction, the y-axis direction, or any other direction.

[0123] Alternatively, the following method may be used as an alternative ordering method: Figures 33 and 34 are diagrams for explaining this processing.

[0124] In the process of limiting face vertex information described using Figure 19 etc., the target node has already completed counterclockwise sorting of edge vertices (E0 to E3) by simple sorting. When face vertex F is inserted into this sorting sequence, it is clear from the distance between face vertex F and each edge vertex that the nearest edge vertices are vertices E1 and E2, so it is clear that face vertex F will be inserted after vertex E1 and before vertex E2. By repeating this for each face vertex, it is possible to perform rotational sorting including edge vertices and face vertices, as shown in Figure 33.

[0125] The decoding device also selects two points from a group of vertices, including edge vertices and face vertices, in order with the centroid vertex at the center, and generates a triangle using the two selected points and the centroid vertex, thereby generating multiple triangles as shown in Figure 34.

[0126] [Syntax] In relation to reducing the amount of data in the bitstream, the information transmitted from the encoding device to the decoding device for restoring face vertices is one bit of information (face vertex information) indicating whether or not to generate a face vertex on each face that is restricted based on geometry.

[0127] 35 is a diagram showing an example of the syntax of GDU (geometry_data_unit_data) included in a bitstream. The GDU includes octree information and geometry_trisoup_data. The geometry_trisoup_data includes the number of edges, edge vertex information, the number of edge vertices, edge vertex position information, the number of transmission faces, and face vertex information.

[0128] The octree information is information that indicates the configuration of a pruned octree, and is information that indicates the positions of multiple leaf nodes included in the pruned octree.

[0129] The number of edges indicates the number of unique edges. A unique edge is a unique edge that excludes edges with overlapping coordinates. Edge vertex information is provided for each edge, and edge vertex information [i] indicates whether an edge vertex exists on the i-th edge. For example, a value of 0 indicates that no edge vertex exists, and a value of 1 indicates that an edge vertex exists.

[0130] The number of edge vertices indicates the number of vertices on an edge, i.e., the number of edge vertices. Edge vertex position information is provided for each edge vertex, and edge vertex position information [i] indicates the position of the i-th edge vertex.

[0131] The number of transmitted faces indicates the total number of face information (face vertex information) to be transmitted. Face vertex information is provided for each face, and face vertex information [i] is 1-bit information indicating whether a face vertex is generated (whether a face vertex exists) on the i-th face. For example, a value of 0 indicates that a face vertex is not generated, and a value of 1 indicates that a face vertex is generated.

[0132] In addition, the number of transmission faces and face vertex information are included in the bit stream when the face vertex function is enabled, and are not included in the bit stream when the face vertex function is not enabled. The face vertex function is a process for generating the face vertices described above.

[0133] For example, a flag indicating whether the face vertex function is enabled or disabled is provided, and whether the face vertex function is enabled or disabled is determined based on this flag. This flag may be stored in, for example, the GPS or GDU header.

[0134] Furthermore, whether or not this face vertex function is enabled may be set for each node. In this case, multiple flags corresponding to multiple nodes may be stored in the GDU header.

[0135] Fig. 36 is a diagram showing a modified example of the syntax of the GDU. The syntax shown in Fig. 36 differs from the syntax shown in Fig. 35 in that the GDU includes face vertex group information instead of the number of transmission faces and face vertex information.

[0136] The face vertex group information indicates, for each of a plurality of faces, whether or not to generate face vertices on that face. In other words, the face vertex group information is information that compiles the plurality of face vertex information shown in Fig. 35. Furthermore, the face vertex group information is included in the bitstream when the face vertex function is enabled, and is not included in the bitstream when the face vertex function is not enabled.

[0137] In the syntax shown in Figure 36, the GDU does not include the number of transmission planes. Here, the node information, edge vertex position information, and centroid vertex position information for applying geometry-based restrictions to each plane are known to the decoding device. Therefore, the decoding device can calculate the number of plane information actually transmitted (the number of transmission planes) by applying geometry-based restrictions to all planes determined from the node adjacency relationships.

[0138] Fig. 37 is a diagram showing a modified example of the syntax of the GDU. The syntax shown in Fig. 37 is an example of syntax when, as described above, a face vertex is generated at a position shifted from the intersection of the face and the line segment connecting two centroid vertices. In the syntax shown in Fig. 37, the GDU further includes an offset amount (x) and an offset amount (y) compared to the syntax shown in Fig. 35.

[0139] An offset amount (x) and an offset amount (y) are set for each face vertex. An offset amount (x) [i] indicates the offset amount in the x-axis direction between the intersection point and the i-th face vertex. An offset amount (y) [i] indicates the offset amount in the y-axis direction between the intersection point and the i-th face vertex. In other words, an offset amount (x) and an offset amount (y) indicate the two-dimensional offset amount from the intersection point to the face vertex.

[0140] For example, the encoding device may quantize the two-dimensional offset amount and store it in the bitstream. In this case, the bitstream includes a quantization parameter used for quantization. The decoding device uses the quantization parameter to dequantize the quantized offset amount included in the bitstream and restore the original offset amount.

[0141] 17, the decoding device generates face vertices when the received face vertex information is true, but face vertices may be generated without relying on the face vertex information. For example, the encoding device may not store the face vertex information in the bitstream, and the decoding device may generate face vertices when a first condition is satisfied and may not generate face vertices when the first condition is not satisfied.

[0142] Also, in the flowchart shown in Figure 16, the encoding device determines the value (true or false) of the face vertex information using the first condition and the second condition, but it may also determine the value of the face vertex information using only one of the first condition and the second condition.

[0143] In the ordering of vertices within a node, the rotation axis passing through the centroid vertex is calculated from the cross product of the normal vector of the plane consisting of the edge vertices and an arbitrary coordinate axis, but the method of calculating the rotation axis for ordering vertices (multiple edge vertices and face vertices) is not limited to this. For example, the vertices may be projected in the direction of an arbitrary axis passing through the centroid vertex, and the axis direction may be determined so that the minimum value of the distance between the projection point and the axis is greater than a predetermined value. Alternatively, the axis direction may be determined so that the sum of the squares of the multiple distances is greater than a predetermined value.

[0144] Furthermore, although a Boolean value (1-bit face vertex information) is transmitted to reduce the data amount of the bitstream, other types of information may be transmitted. For example, the face vertex information may indicate three or more values. For example, a value of 0 may mean "do not generate face vertices," a value of 1 may mean "generate face vertices," and a value of 2 may mean "generate face vertices according to the capabilities of the decoding device." Alternatively, when a Boolean value is used and the Boolean value is true, the decoding device may determine whether to generate face vertices according to the capabilities of the decoding device.

[0145] (Embodiment 2) In this embodiment, a method for omitting transmission of face vertex information will be described. Fig. 38 is a diagram showing an example of the configuration of a bit stream according to this embodiment. As shown in Fig. 38, the bit stream includes an SPS, a GPS, an APS, a GDU (Geom), and an ADU (Attr).

[0146] SPS (Sequence Parameter Set) is metadata (parameter set) common to multiple frames. APS (Attribute Parameter Set) is metadata (parameter set) related to encoding of attribute information. GPS (Geometry Parameter Set) is metadata (parameter set) related to encoding of location information. For example, APS and GPS are metadata common to multiple frames.

[0147] A GDU is a data unit (geometry data unit) of coded data of position information, and an ADU is a data unit (attribute data unit) of coded data of attribute information.

[0148] In FIG. 38, each point has two types of attribute information (Attr(0) and Attr(1)). Also, GDUs and ADUs are generated for each processing unit, each of which includes multiple 3D points. A processing unit is, for example, a frame or a slice.

[0149] 38 shows example configurations of GDUs in a comparative example (when the method of this embodiment is not applied) and in this embodiment. In the comparative example, the GDU includes a GDU header, octree data, edge vertex data, centroid vertex data, face vertex data, and a GDU footer. The GDU header is the header (control information) of the GDU. The octree data is information indicating the octree structure, and includes, for example, the octree information shown in FIG. 35. The edge vertex data is information related to edge vertices, and includes, for example, the number of edges, edge vertex information, edge vertex count, and edge vertex position information shown in FIG. 35.

[0150] The centroid vertex data is information about the centroid vertex, and includes, for example, information indicating the position of the centroid vertex. The face vertex data is information about the face vertex, and includes, for example, the number of transmission faces and face vertex information shown in Fig. 35. The GDU footer is the footer (control information) of the GDU.

[0151] 38, in this embodiment, the bitstream includes, for example, a face vertex non-transmission flag in the GDU header. The face vertex non-transmission flag is, for example, a 1-bit flag that indicates whether the bitstream includes information about face vertices (face vertex data).

[0152] When the encoding device has completed the process of determining whether or not a face vertex exists for each eligible face, and if face vertices exist on all eligible faces in the processing unit (e.g., slice) corresponding to the GDU, the encoding device stores a face vertex non-transmission flag indicating a value of 1 in the GDU header. In other words, the face vertex non-transmission flag indicates whether or not face vertices exist on all eligible faces. In this case, the bitstream does not include face vertex data. In other words, transmission of the face vertex data is omitted.

[0153] The qualified surface is a surface that is determined to satisfy the first condition in step S221 shown in FIG. 16 or step S231 shown in FIG.

[0154] On the other hand, if the face vertex non-transmission flag is 0, a bitstream containing face vertex data is transmitted.

[0155] Furthermore, in processing each processing unit (e.g., slice), the decoder determines that face vertices exist on all eligible faces if the face vertex non-transmission flag is 1. In this case, the decoder skips reading of face vertex data and arithmetic decoding of the face vertex data.

[0156] If the face vertex non-transmission flag is 0, the decoding device reads the face vertex data and arithmetically decodes the face vertex data to obtain information on the face vertices of each eligible face.

[0157] The above process reduces the amount of data transmitted without compromising the reconstructed shape of the Trithorpe triangle. Furthermore, the process can be switched depending on the face vertex non-transmission flag, shortening the decoding process time.

[0158] The face vertex non-transmission flag may be stored in a location other than the GDU header. For example, the face vertex non-transmission flag may be stored at any position after the GDU header. For example, the face vertex non-transmission flag may be stored immediately after the centroid vertex data and immediately before the face vertex data.

[0159] The face vertex non-transmission flag may also be stored in a sequence header (for example, SPS or GPS). In this case, the face vertex non-transmission flag stored in the sequence header may be used in common by multiple processing units (GDUs).

[0160] Next, the flow of the encoding process according to this embodiment will be described. The encoding process according to this embodiment differs from the encoding process shown in Fig. 14 in that steps S203 and S204 are replaced with steps S203A and S204A.

[0161] Figure 39 is a flowchart of this face vertex information transmission process (S203A and S204A). The process shown in Figure 39 differs from steps S203 and S204 shown in Figure 16 in that steps S251 to S254 are added instead of step S227. The following will mainly explain the differences from the process shown in Figure 16.

[0162] After completing the loop process for each face, the encoding device determines whether all faces that satisfy the first condition also satisfy the second condition (S251). If at least some of the faces that satisfy the first condition do not satisfy the second condition (No in S251), the encoding device sets the face vertex non-transmission flag to 0 and stores the face vertex non-transmission flag with a value of 0 in the bitstream (S252). Next, the encoding device encodes the accumulated face vertex information and stores the encoded face vertex information in the bitstream (S254).

[0163] On the other hand, if all of the faces that satisfy the first condition also satisfy the second condition (Yes in S251), the encoding device sets the face vertex non-transmission flag to value 1 and stores the face vertex non-transmission flag with value 1 in the bitstream (S253). In this case, the encoding device does not encode the accumulated pieces of face vertex information, and does not store the encoded face vertex information in the bitstream.

[0164] In addition, if all faces that satisfy the first condition also satisfy the second condition (Yes in S251), the encoding device may, for example, set the face vertex non-transmission flag to the value 0 and store the face vertex information in the bitstream, depending on other conditions.

[0165] Next, the flow of the decoding process according to this embodiment will be described. The decoding process according to this embodiment differs from the decoding process shown in Fig. 15 in that steps S213 and S214 are replaced with steps S213A and S214A.

[0166] Figure 40 is a flowchart of the face vertex information decoding process (S213A and S214A). The process shown in Figure 40 differs from the process of steps S213 and S214 shown in Figure 17 in that steps S261 to S262 have been added. The following mainly describes the differences from the process shown in Figure 17.

[0167] If the first condition is met (Yes in S231), the decoding device acquires (decodes) the face vertex non-transmission flag stored in the bitstream and determines whether the face vertex non-transmission flag is 1 (S261). If the face vertex non-transmission flag is 0 (No in S261), the decoding device decodes face vertex information from the bitstream that indicates whether or not to generate face vertices on the target face (S232). This determines whether or not to generate face vertices on the target face ("true" or "false").

[0168] On the other hand, if the face vertex non-transmission flag is 1 (Yes in S261), the decoding device does not decode the face vertex information of the target surface from the bitstream, and sets the face vertex information of the target surface to "true" (S262).

[0169] Next, the decoding device generates face vertices on the target surface based on the face vertex information (true / false) (S234). That is, if the face vertex information is "true", the decoding device generates face vertices on the target surface, and if the face vertex information is "false", the decoding device does not generate face vertices on the target surface.

[0170] Next, an example of the syntax of information stored in the bitstream according to this embodiment will be described. Fig. 41 is a diagram showing an example of the syntax of the GDU header (geometry_data_unit_header) and GDU (geometry_data_unit_data) included in the bitstream. The syntax shown in Fig. 41 differs from the syntax shown in Fig. 35 in that the GDU header includes a face vertex valid flag and a face vertex non-transmission flag.

[0171] The face vertex valid flag indicates whether the face vertex function is valid or invalid. The face vertex function is the process of generating the face vertices described above. For example, if the face vertex valid flag is set to a value of 1, the face vertex function is valid (face vertices are generated), and if the face vertex valid flag is set to a value of 0, the face vertex function is invalid (face vertices are not generated).

[0172] The face vertex non-transmission flag is stored in the GDU header, for example, when the face vertex valid flag = 1 (face vertex function is enabled), and is not stored in the GDU header when the face vertex valid flag = 0 (face vertex function is disabled).

[0173] The face vertex non-transmission flag is a flag that indicates that face vertices should be created for all of the geometrically restricted faces (eligible faces). In other words, the face vertex non-transmission flag is associated with each of the faces that satisfy the first condition, and indicates whether or not 1-bit information (face vertex information) indicating whether or not to generate a face vertex for that face has been transmitted.

[0174] Furthermore, when the face vertex valid flag is 1 and the face vertex non-transmission flag is 0, the GUD includes information for generating face vertices (the number of transmission faces and face vertex information [i]). The meaning of this information is the same as in Fig. 35. Furthermore, the information for generating face vertices is not included in the bitstream unless the face vertex valid flag is 1 and the face vertex non-transmission flag is 0.

[0175] Fig. 42 is a diagram showing a modified example of the GDU header and GDU syntax. The syntax shown in Fig. 42 differs from the syntax shown in Fig. 41 in that the GDU includes face vertex group information instead of the number of transmission faces and face vertex information. The syntax shown in Fig. 42 also differs from the syntax shown in Fig. 36 in that the GDU header includes a face vertex valid flag and a face vertex non-transmission flag. The meanings of these flags are the same as in Fig. 41.

[0176] Furthermore, when the face vertex valid flag is 1 and the face vertex non-transmission flag is 0, the GUD includes information for generating face vertices (face vertex group information). The meaning of the face vertex group information is the same as in Fig. 36. Furthermore, the information for generating face vertices is not included in the bitstream unless the face vertex valid flag is 1 and the face vertex non-transmission flag is 0.

[0177] The face vertex valid flag and the face vertex non-transmission flag may be stored in the GDU header as shown in Figures 41 and 42, or may be stored at another position in the bitstream. For example, these flags may be stored in a sequence header (e.g., SPS or GPS). Also, these flags may be located at any position after the GDU header and before the face vertex information. Also, the face vertex valid flag and the face vertex non-transmission flag may be stored at a different position.

[0178] Fig. 43 is a diagram showing modified examples of the syntax of a GDU header and a GDU. For example, the syntax shown in Fig. 43 differs from the syntax shown in Fig. 41 in the location of the face vertex non-transmission flag. For example, as shown in Fig. 43, the face vertex non-transmission flag may be placed before information for generating face vertices in a GDU (the number of transmission faces and face vertex information [i]).

[0179] [Summary] As described above, the decoding device (3D data decoding device) according to the embodiment performs the processing shown in FIG. 44 . The decoding device receives a bitstream including position information (e.g., octree information) indicating multiple nodes constituting an octree structure and first control information (e.g., face vertex information or face vertex group information) indicating whether a first face of a first node included in the multiple nodes has first face vertices provided on a side other than the multiple first edges of the first face (S301), and generates or does not generate first face vertices on the first face according to the first control information (S302). Each of the multiple nodes is a unit including multiple 3D points. The first node includes a first centroid vertex and multiple first edge vertices used in the Trithorpe method. The first face vertex, the first centroid vertex, and the multiple first edge vertices define a triangle (e.g., a Trithorpe triangle) in which the 3D points in the first node are arranged. The first control information is provided for a face of the first node that satisfies a predetermined condition (e.g., a first condition). That is, the bitstream includes first control information for aspects that satisfy the predetermined conditions, but does not include first control information for aspects that do not satisfy the predetermined conditions.

[0180] According to this, by providing the first control information for the planes that satisfy the predetermined conditions, the amount of data in the bitstream is reduced compared to when the first control information is provided for all planes, and therefore the amount of processing by the decoding device is reduced.

[0181] For example, the predetermined conditions include a first condition that the face includes two or three edge vertices, whereby when the possibility of generating a face vertex is low, generation of the first control information is omitted, thereby reducing the amount of data in the bitstream.

[0182] For example, the predetermined condition includes a second condition that the first vector, the second vector, and the third vector face in the same direction. The first vector is a vector from a first center of the plurality of first edge vertices to a first centroid vertex. The second vector is a vector from a second center of the plurality of second edge vertices of a second node to a second centroid vertex of the second node, the second node being adjacent to the first node so that the face is tangent to the second node. The third vector is a vector from a first line connecting two edge vertices of the face to a virtual face vertex, the virtual face vertex being located at a position where a second line connecting the first centroid vertex and the second centroid vertex intersects with the face. This omits generation of the first control information when the face vertex is unlikely to be generated, thereby reducing the amount of data in the bitstream.

[0183] For example, if the dot product of the first vector and the third vector is positive and the dot product of the second vector and the third vector is positive, it is determined that the first vector, the second vector, and the third vector are oriented in the same direction. This allows the decoding device to appropriately determine the second condition.

[0184] For example, the predetermined conditions include a third condition of whether an adjacent node adjacent to the first node has a centroid vertex, whereby when the possibility of generating a face vertex is low, generation of the first control information is omitted, thereby reducing the amount of data in the bitstream.

[0185] For example, whether or not a first surface indicated by the first control information has a first face vertex is determined depending on whether or not the number or density of points contained in an area within a predetermined distance from the intersection of (i) a line segment connecting the first centroid vertex and the second centroid vertex of a second node adjacent to the first node, and (ii) a surface shared by the first node and the second node is equal to or greater than a predetermined threshold. This allows for an appropriate setting of whether or not to generate a face vertex.

[0186] For example, the bitstream further includes second control information (e.g., a face vertex non-transmission flag) indicating whether or not the first control information for the multiple faces of the first node is included in the bitstream, whereby the decoding device can switch whether or not to perform the face vertex generation process using the first control information by referring to the second control information.

[0187] For example, when the second control information indicates that the first control information for multiple faces of the first node is included in the bitstream (e.g., the face vertex non-transmission flag = 0), the decoding device generates or does not generate face vertices for the target face to be processed according to the first control information corresponding to the target face among the multiple faces.When the second control information indicates that the first control information for multiple faces of the first node is not included in the bitstream (e.g., the face vertex non-transmission flag = 1), the decoding device generates face vertices for each of the multiple faces.

[0188] According to this, when face vertices are generated for each of a plurality of faces, the bitstream does not need to include the first control information, thereby reducing the data amount of the bitstream.

[0189] For example, if all of the faces of the first node that satisfy a predetermined condition also satisfy another predetermined condition, the second control information indicates that the first control information for the faces of the first node is not included in the bitstream. Accordingly, for example, if face vertices are generated for each of the faces that satisfy the predetermined condition, the bitstream does not need to include the first control information. Therefore, the amount of data in the bitstream can be reduced.

[0190] 45 is a block diagram of the decoding device 10. For example, the decoding device 10 includes a processor 11 and a memory 12, and the processor 11 performs the above-described processing using the memory 12.

[0191] Furthermore, an encoding device (three-dimensional data encoding device) according to an embodiment performs the process shown in FIG. 46 . The encoding device generates position information indicating multiple nodes constituting an octree structure (S311), generates first control information indicating whether a first face of a first node included in the multiple nodes has first face vertices provided on a side other than the multiple first edges of the first face (S312), and generates a bitstream including the position information and the first control information (S313). Each of the multiple nodes is a unit including multiple three-dimensional points. The first node includes a first centroid vertex and multiple first edge vertices used in the Trithorpe method. The first face vertex, the first centroid vertex, and the multiple first edge vertices define a triangle (e.g., a Trithorpe triangle) in which the three-dimensional points in the first node are arranged. The first control information is provided for faces of the first node that satisfy a predetermined condition (e.g., a first condition). In other words, the encoding device generates first control information for faces that satisfy the predetermined condition and does not generate first control information for faces that do not satisfy the predetermined condition.

[0192] According to this, by providing the first control information for the planes that satisfy the predetermined conditions, the amount of data in the bitstream is reduced compared to when the first control information is provided for all planes.

[0193] For example, the predetermined conditions include a first condition that the face includes two or three edge vertices, whereby generation of the first control information is omitted when the possibility of generating a face vertex is low.

[0194] For example, the predetermined condition includes a second condition that the first vector, the second vector, and the third vector face in the same direction. The first vector is a vector from a first center of the plurality of first edge vertices to a first centroid vertex. The second vector is a vector from a second center of the plurality of second edge vertices of a second node to a second centroid vertex of the second node, the second node being adjacent to the first node so that the face is tangent to the second node. The third vector is a vector from a first line connecting two edge vertices of the face to a virtual face vertex, the virtual face vertex being located at a position where a second line connecting the first centroid vertex and the second centroid vertex intersects with the face. This allows generation of the first control information to be omitted when the possibility of generating a face vertex is low.

[0195] For example, if the dot product of the first vector and the third vector is positive and the dot product of the second vector and the third vector is positive, it is determined that the first vector, the second vector, and the third vector are oriented in the same direction. This allows the encoding device to appropriately determine the second condition.

[0196] For example, the predetermined conditions include a third condition of whether an adjacent node adjacent to the first node has a centroid vertex, whereby generation of the first control information is omitted when the possibility of generating a face vertex is low.

[0197] For example, the encoding device determines whether the first face has a first face vertex based on whether the number or density of points contained in an area within a predetermined distance from the intersection of (i) a line segment connecting a first centroid vertex and a second centroid vertex of a second node adjacent to the first node, and (ii) a face shared by the first node and the second node is equal to or greater than a predetermined threshold. This allows the encoding device to appropriately determine whether to generate a face vertex.

[0198] For example, the bitstream further includes second control information (e.g., a face vertex non-transmission flag) indicating whether or not the first control information for the multiple faces of the first node is included in the bitstream, whereby the decoding device can switch whether or not to perform the face vertex generation process using the first control information by referring to the second control information.

[0199] For example, if all of the faces of the first node that satisfy a predetermined condition satisfy another predetermined condition, the encoding device generates second control information indicating that the first control information for the faces of the first node is not included in the bitstream. Accordingly, for example, if face vertices are generated for each of all faces that satisfy the predetermined condition, the bitstream does not need to include the first control information. Therefore, the amount of data in the bitstream can be reduced.

[0200] 47 is a block diagram of the encoding device 20. For example, the encoding device 20 includes a processor 21 and a memory 22, and the processor 21 uses the memory 22 to perform the above-described processing.

[0201] The above describes the encoding device (three-dimensional data encoding device) and decoding device (three-dimensional data decoding device) according to the embodiments and modifications of the present disclosure, but the present disclosure is not limited to these embodiments.

[0202] Furthermore, each processing unit included in the encoding device, decoding device, etc. according to the above-described embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0203] Furthermore, the integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI may also be used.

[0204] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0205] Furthermore, the present disclosure may be realized as an encoding method (three-dimensional data encoding method) or a decoding method (three-dimensional data decoding method) executed by an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device), etc.

[0206] The present disclosure may also be realized as a program that causes a computer, processor, or device to execute the encoding method or decoding method. The present disclosure may also be realized as a bitstream generated by the encoding method. The present disclosure may also be realized as a recording medium on which the program or the bitstream is recorded. For example, the present disclosure may also be realized as a non-transitory computer-readable recording medium on which the program or the bitstream is recorded.

[0207] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0208] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0209] While the encoding device and decoding device according to one or more aspects have been described based on the embodiments, the present disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the present embodiments and configurations constructed by combining components of different embodiments may also be included within the scope of one or more aspects.

[0210] The present disclosure is applicable to encoding devices and decoding devices.

[0211] 10 Decoding device 11, 21 Processor 12, 22 Memory 20 Encoding device 101 Object space 102, 133 Point cloud 103, 111, 132 Point 104 Leaf node 112 Edge vertex 113, 122 Range 121 Approximation plane 123 Vertex information 131 Triangle 151 Centroid vertex 161 Face vertex

Claims

1. A decoding method comprising: receiving a bit stream including position information indicating a plurality of nodes constituting an octree structure and first control information indicating whether a first face of a first node included in the plurality of nodes has a first face vertex provided other than a plurality of first edges of the first face; generating or not generating the first face vertex on the first face in accordance with the first control information; each of the plurality of nodes being a unit including a plurality of three-dimensional points; the first node including a first centroid vertex and a plurality of first edge vertices used in a Trisope method; the first face vertex, the first centroid vertex and the plurality of first edge vertices defining a triangle in which the three-dimensional points in the first node are arranged; and the first control information being provided for a face of the first node that satisfies a predetermined condition.

2. The decoding method according to claim 1, wherein the predetermined conditions include a first condition that the face includes two or three edge vertices.

3. The decoding method according to claim 1, wherein the predetermined conditions include a second condition that the first vector, the second vector, and the third vector face in the same direction, the first vector being a vector from a first center of the plurality of first edge vertices to the first centroid vertex, the second vector being a vector from a second center of the plurality of second edge vertices of a second node to a second centroid vertex of the second node, the second node being adjacent to the first node such that the face is tangent to the second node, and the third vector being a vector from a first line connecting two edge vertices of the face to a virtual face vertex, and the virtual face vertex is located at a position where a second line connecting the first centroid vertex and the second centroid vertex intersects with the face.

4. The decoding method according to claim 3, wherein if the dot product of the first vector and the third vector is positive and the dot product of the second vector and the third vector is positive, it is determined that the first vector, the second vector, and the third vector point in the same direction.

5. The decoding method according to claim 1, wherein the predetermined conditions include a third condition that an adjacent node adjacent to the first node has a centroid vertex.

6. The decoding method according to claim 1, wherein whether or not the first surface indicated by the first control information has a first face vertex is determined depending on whether or not the number or density of points contained in an area within a predetermined distance from the intersection of (i) a line segment connecting the first centroid vertex and a second centroid vertex of a second node adjacent to the first node, and (ii) a surface shared by the first node and the second node is equal to or greater than a predetermined threshold.

7. The method of decoding of claim 1, wherein the bitstream further includes second control information indicating whether the first control information for multiple faces of the first node is included in the bitstream.

8. The decoding method of claim 7, further comprising: generating or not generating face vertices for a target face among the multiple faces in accordance with the first control information corresponding to the target face among the multiple faces when the second control information indicates that the first control information for the multiple faces of the first node is included in the bitstream; and generating face vertices for each of the multiple faces when the second control information indicates that the first control information for the multiple faces of the first node is not included in the bitstream.

9. The decoding method of claim 7, wherein if all of the multiple faces of the first node that satisfy the predetermined condition satisfy another predetermined condition, the second control information indicates that the first control information for the multiple faces of the first node is not included in the bitstream.

10. An encoding method comprising: generating position information indicating a plurality of nodes constituting an octree structure; generating first control information indicating whether a first face of a first node included in the plurality of nodes has a first face vertex provided other than a plurality of first edges of the first face; generating a bit stream including the position information and the first control information; each of the plurality of nodes is a unit including a plurality of three-dimensional points; the first node includes a first centroid vertex and a plurality of first edge vertices used in a Trisope method; the first face vertex, the first centroid vertex and the plurality of first edge vertices define a triangle in which the three-dimensional points in the first node are arranged; and the first control information is provided for a face of the first node that satisfies a predetermined condition.

11. A decoding device for decoding a plurality of three-dimensional points, comprising: a processor; and a memory, wherein the processor uses the memory to receive a bit stream including position information indicating a plurality of nodes constituting an octree structure and first control information indicating whether a first face of a first node included in the plurality of nodes has a first face vertex provided other than a plurality of first edges of the first face, and generates or does not generate the first face vertex on the first face in accordance with the first control information, each of the plurality of nodes being a unit including a plurality of three-dimensional points, the first node including a first centroid vertex and a plurality of first edge vertices used in a trisope method, the first face vertex, the first centroid vertex, and the plurality of first edge vertices define a triangle in which the three-dimensional points in the first node are arranged, and the first control information is provided for a face of the first node that satisfies a predetermined condition.

12. An encoding device for encoding a plurality of three-dimensional points, comprising: a processor; and a memory, wherein the processor uses the memory to generate position information indicating a plurality of nodes constituting an octree structure; generate first control information indicating whether a first face of a first node included in the plurality of nodes has a first face vertex provided other than the plurality of first edges of the first face; generate a bit stream including the position information and the first control information; each of the plurality of nodes is a unit including a plurality of three-dimensional points; the first node includes a first centroid vertex and a plurality of first edge vertices used in a trisope method; the first face vertex, the first centroid vertex, and the plurality of first edge vertices define a triangle in which the three-dimensional points in the first node are arranged; and the first control information is provided for a face of the first node that satisfies a predetermined condition.