Point cloud decoding device, point cloud decoding method, and program

Abstract

To improve coding efficiency.SOLUTION: A point cloud decoding device 200 includes an approximate surface synthesis unit 2030 for decoding, for each node of a predetermined size, a node size that applies Trisoup to descendant nodes obtained by recursively partitioning the node by Octree.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a point cloud decoding device, a point cloud decoding method, and a program.

Background Art

[0002] In Non-Patent Document 1, a geometric information encoding technique called Trisoup is disclosed.

Prior Art Document

Non-Patent Document

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the method of Non-Patent Document 1 has a problem that Trisoup can only be executed with a fixed node size for each slice.

[0005] Therefore, the present invention has been made in view of the above problems, and in Trisoup with a plurality of node sizes, by decoding the Trisoup node size for each region, Trisoup nodes of the same node size are locally adjacent, so that it becomes easy to utilize the correlation in the spatial direction, and an object of the present invention is to provide a point cloud decoding device, a point cloud decoding method, and a program capable of improving the encoding efficiency.

Means for Solving the Problems

[0006] A first feature of the present invention is a point cloud decoding device, which includes an approximate surface synthesis unit that decodes a node size to which Trisoup is applied to descendant nodes obtained by recursively dividing the node by an Octree for each node of a predetermined size, and this is the gist.

[0007] The second feature of the present invention is a point cloud decoding method that includes a step of decoding the node size to which Trisoup is applied to descendant nodes obtained by recursively dividing each node of a predetermined size using Octree.

[0008] A third feature of the present invention is a program that causes a computer to function as a point cloud decoder, wherein the point cloud decoder includes an approximate surface synthesis unit that decodes the node size to which Trisoup is applied to descendant nodes obtained by recursively dividing each node of a predetermined size using Octree. [Effects of the Invention]

[0009] According to the present invention, in Trisoup with multiple node sizes, by decoding the Trisoup node size for each region, local Trisoup nodes of the same node size are adjacent, making it easier to utilize spatial correlation and improve coding efficiency. This provides a point cloud decoding device, point cloud decoding method, and program. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows an example of the configuration of a point cloud processing system 10 according to one embodiment. [Figure 2] Figure 2 shows an example of the functional block of a point cloud decoding device 200 according to one embodiment. [Figure 3] Figure 3 shows an example of the configuration of encoded data (bitstream) received by the geometric information decoding unit 2010 of a point cloud decoding device 200 according to one embodiment. [Figure 4] Figure 4 shows an example of the syntax configuration of GPS2011. [Figure 5] Figure 5 shows an example of the syntax configuration of GSH2012. [Figure 6] Figure 6 shows an example of the syntax configuration of GSH2012. [Figure 7] FIG. 7 is a flowchart showing an example of processing in the tree synthesis unit 2020 of the point cloud decoding apparatus 200 according to an embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of processing in the approximate surface synthesis unit 2030 of the point cloud decoding apparatus 200 according to an embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the processing of step S703 shown in FIG. 8. [Figure 10] FIG. 10 is a flowchart showing an example of the processing of step S704 shown in FIG. 8. [Figure 11] FIG. 11 is a flowchart showing a specific example of a method for generating mask information. [Figure 12] FIG. 12 is a diagram showing an example of generation of segments and mask values. [Figure 13] FIG. 13 is a flowchart showing an example of the processing of step S705 shown in FIG. 8. [Figure 14] FIG. 14 is a flowchart showing an example of the processing of step S706 shown in FIG. 8. [Figure 15] FIG. 15 is a flowchart showing an example of the processing of step S708 shown in FIG. 8. [Figure 16A] FIG. 16A is a diagram for explaining an example of the processing of step S705 in FIG. 8. [Figure 16B] FIG. 16B is a diagram for explaining an example of the processing of step S706 in FIG. 8. [Figure 16C] FIG. 16C is a diagram for explaining an example of the processing of step S708 in FIG. 8. [Figure 17] FIG. 17 is a flowchart showing an example of the processing of step S709 shown in FIG. 8. [Figure 18] FIG. 18 is a flowchart showing an example of the processing of step S706 shown in FIG. 8. [Figure 19] FIG. 19 is a flowchart showing an example of processing in the approximate surface synthesis unit 2030 of the point cloud decoding apparatus 200 according to an embodiment. [Figure 20] Figure 20 is a diagram illustrating an example of the processing of the approximate surface synthesis unit 2030 of the point cloud decoding device 200 according to one embodiment. [Figure 21] Figure 21 is a diagram showing an example of the functional blocks of the point cloud coding device 100 according to this embodiment. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described below with reference to the drawings. Note that the components in the following embodiments can be replaced with existing components as appropriate, and various variations are possible, including combinations with other existing components. Therefore, the description of the following embodiments does not limit the content of the invention as described in the claims.

[0012] (First Embodiment) The point cloud processing system 10 according to the first embodiment of the present invention will be described below with reference to Figures 1 to 21. Figure 1 is a diagram showing the point cloud processing system 10 according to the embodiment of this invention.

[0013] As shown in Figure 1, the point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0014] The point cloud encoding device 100 is configured to generate encoded data (bitstream) by encoding the input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bitstream.

[0015] The input and output point cloud signals consist of positional and attribute information for each point within the point cloud. Attribute information includes, for example, the color and reflectance of each point.

[0016] Here, such a bitstream may be transmitted from the point cloud encoding device 100 to the point cloud decoding device 200 via a transmission line. Alternatively, the bitstream may be stored in a storage medium and then provided from the point cloud encoding device 100 to the point cloud decoding device 200.

[0017] (Point cloud decoder 200) The point cloud decoder 200 according to this embodiment will be described below with reference to Figure 2. Figure 2 is a diagram showing an example of the functional block of the point cloud decoder 200 according to this embodiment.

[0018] As shown in Figure 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, an RAHT unit 2080, an LoD calculation unit 2090, an inverse lifting unit 2100, and an inverse color transformation unit 2110.

[0019] The geometric information decoding unit 2010 is configured to take the bitstream related to geometric information (geometric information bitstream) from the bitstream output from the point cloud coding device 100 as input and decode the syntax.

[0020] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) to control the decoding process of the location information.

[0021] The tree synthesis unit 2020 is configured to generate tree information indicating which regions within the decoding target space contain points, by taking as input the control data decoded by the geometric information decoding unit 2010 and the occupancy code, which indicates which node in the tree (described later) contains the point cloud.

[0022] The system may also be configured to perform the decoding of the occupancy code within the tree synthesis unit 2020.

[0023] This process divides the space to be decoded into rectangular prisms, determines whether a point exists within each rectangular prism by referring to the occupancy code, divides the rectangular prism containing a point into multiple rectangular prisms, and recursively repeats the process of referring to the occupancy code, thereby generating tree information.

[0024] In this case, interpretation may be used when decoding such occupancy code.

[0025] In this embodiment, a method called "Octree" can be used, which recursively performs octree partitioning by always treating the cuboid as a cube, and a method called "QtBt" can be used, which performs quadtree partitioning and binary partitioning in addition to octree partitioning. Whether or not to use "QtBt" is transmitted as control data from the point cloud encoding device 100.

[0026] In this embodiment, the method of decoding geometric information by dividing the aforementioned cube into 1x1x1 size parts using "Octree" is specifically referred to as "Octree only".

[0027] Furthermore, even when using "Octree" in combination with "QtBt," the method of decoding the geometric information by dividing the aforementioned rectangular prism into 1x1x1 size using only "Octree" and "QtBt" can also be called "Octree only."

[0028] Alternatively, if the control data specifies that predictive coding should be used, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree configuration determined by the point cloud coding device 100.

[0029] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using tree information generated by the tree synthesis unit 2020, and to decode the point cloud based on this approximate surface information.

[0030] Approximate surface information is used, for example, when decoding 3D point cloud data of an object, in cases where the point cloud is densely distributed on the object's surface. Instead of decoding each individual point cloud, the region where the point cloud exists is approximated and represented by a small plane.

[0031] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information and decode point clouds using a method called "Trisoup," for example. A specific example of the "Trisoup" process will be described later. Furthermore, this process can be omitted when decoding sparse point clouds acquired by Lidar or the like.

[0032] The geometric information reconstruction unit 2040 is configured to reconstruct the geometric information (position information in the coordinate system assumed by the decoding process) of each point in the point cloud data to be decoded, based on the tree information generated by the tree synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0033] The inverse coordinate transformation unit 2050 is configured to take the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform it from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

[0034] The attribute information decoding unit 2060 is configured to take the bitstream related to attribute information (attribute information bitstream) from the bitstream output from the point cloud coding device 100 as input and decode the syntax.

[0035] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) to control the decoding process of attribute information.

[0036] Furthermore, the attribute information decoding unit 2060 is configured to decode quantized residual information from the decoded syntax.

[0037] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and the quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, in order to generate inverse quantized residual information.

[0038] The inversely quantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090, depending on the characteristics of the point cloud to be decoded. Which unit it is output to is specified by the control data decoded by the attribute information decoding unit 2060.

[0039] The RAHT unit 2080 is configured to take the inversely quantized residual information generated by the inverse quantization unit 2070 and the geometric information reconstruction unit 2040 as input, and decode the attribute information of each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform) (inverse Haar transform in the decoding process). As a specific example of the RAHT processing, the method described in Non-Patent Document 1 can be used.

[0040] The LoD calculation unit 2090 is configured to take geometric information generated by the geometric information reconstruction unit 2040 as input and generate LoD (Level of Detail).

[0041] LoD (Level of Data) is information used to define reference relationships (referring points and referenced points) for implementing predictive coding, which involves predicting the attribute information of one point from the attribute information of another point and then encoding or decoding the prediction residual.

[0042] In other words, LoD is information that defines a hierarchical structure in which each point included in geometric information is classified into multiple levels, and the attributes of points belonging to lower levels are encoded or decoded using the attribute information of points belonging to higher levels.

[0043] As for a specific method for determining the LoD, for example, the method described in Non-Patent Document 1 above may be used.

[0044] The inverse lifting unit 2100 is configured to decode the attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inversely quantized residual information generated by the inverse quantization unit 2070. As a specific processing method for inverse lifting, for example, the method described in Non-Patent Document 1 above can be used.

[0045] The reverse color conversion unit 2110 is configured to perform reverse color conversion on the attribute information output from the RAHT unit 2080 or the reverse lifting unit 2100 if the attribute information to be decoded is color information and color conversion has been performed on the point cloud encoding device 100 side. Whether or not such reverse color conversion processing is performed is determined by the control data decoded by the attribute information decoding unit 2060.

[0046] The point cloud decoder 200 is configured to decode and output attribute information for each point in the point cloud through the above processing.

[0047] (Geometric information decoding unit 2010) The control data decoded by the geometric information decoding unit 2010 will be explained below using Figures 3 to 6.

[0048] Figure 3 shows an example of the configuration of encoded data (bitstream) received by the geometric information decoding unit 2010.

[0049] Firstly, the bitstream may include GPS2011. GPS2011, also known as the geometry parameter set, is a set of control data related to the decoding of geometric information. Specific examples will be discussed later. Each GPS2011 includes at least GPS ID information to identify each individual GPS2011 if multiple GPS2011s exist.

[0050] Secondly, the bitstream may include GSH2012A / 2012B. GSH2012A / 2012B, also called geometry slice headers or geometry data unit headers, are sets of control data corresponding to slices, which will be described later. Hereafter, we will use the term "slice," but you can also read "slice" as "data unit." Specific examples will be given later. Each GSH2012A / 2012B includes at least GPS ID information to specify the GPS2011 corresponding to each GSH2012A / 2012B.

[0051] Thirdly, the bitstream may include slice data 2013A / 2013B after GSH2012A / 2012B. Slice data 2013A / 2013B contains data that encodes geometric information. An example of slice data 2013A / 2013B is the occupancy code, which will be discussed later.

[0052] As described above, the bitstream is configured so that each slice data 2013A / 2013B corresponds to one GSH2012A / 2012B and one GPS2011.

[0053] As described above, GSH2012A / 2012B allows you to specify which GPS2011 to refer to using GPS ID information, so a common GPS2011 can be used for multiple slice data 2013A / 2013B.

[0054] In other words, GPS2011 does not necessarily need to be transmitted for each slice. For example, as shown in Figure 3, the bitstream can be configured so that GPS2011 is not encoded immediately before GSH2012B and slice data 2013B.

[0055] Note that the configuration in Figure 3 is merely an example. As long as each slice data 2013A / 2013B corresponds to GSH2012A / 2012B and GPS2011, other elements besides those mentioned above may be added as components of the bitstream.

[0056] For example, as shown in Figure 3, the bitstream may include a sequence parameter set (SPS) 2001. Similarly, it may be formatted to a different configuration than that shown in Figure 3 during transmission. Furthermore, it may be combined with the bitstream decoded by the attribute information decoding unit 2060 described later and transmitted as a single bitstream.

[0057] Figure 4 shows an example of the GPS2011 syntax configuration.

[0058] Please note that the syntax names described below are merely examples. If the functionality of the syntax described below is the same, the syntax names may differ.

[0059] GPS2011 may include GPS ID information (gps_geom_parameter_set_id) to identify each GPS2011.

[0060] In Figure 4, the Descriptor column indicates how each syntax is encoded. ue(v) means it is an unsigned zero-order exponential Golomb code, and u(1) means it is a 1-bit flag.

[0061] GPS2011 may include a flag (trisoup_enabled_flag) that controls whether or not to use Trisoup in the approximate surface synthesis unit 2030.

[0062] For example, you could define that Trisoup is not used when the value of trisoup_enabled_flag is "0", and that Trisoup is used when the value of trisoup_enabled_flag is "1".

[0063] The geometric information decoding unit 2020 may be configured to decode the following syntax in addition when using Trisoup, that is, when the value of trisoup_enabled_flag is "1".

[0064] Note that trisoup_enabled_flag may be included in SPS2001 instead of GPS2011.

[0065] GPS2011 may include a flag (trisoup_multilevel_enabled_flag, first flag) that controls whether or not to allow multilevel trisoup.

[0066] For example, you could define that when the value of trisoup_multilevel_enabled_flag is "0", multilevel Trisoup is not allowed, meaning only single-level Trisoup is performed, and when the value of trisoup_multilevel_enabled_flag is "1", multilevel Trisoup is allowed.

[0067] If the syntax in question is not included in GPS2011, the value of that syntax may be treated as the value for performing a single-level Trisoup, i.e., "0".

[0068] Note that trisoup_multilevel_enabled_flag may be defined to be included in SPS2001 instead of GPS2011. In this case, if trisoup_multilevel_enabled_flag is not included in SPS2001, the value of the syntax may be considered as the value for performing Trisoup at a single level, i.e., "0".

[0069] Figures 5 and 6 show an example of the syntax configuration of GSH2012. As mentioned above, GSH is also known as GDUH (Geometry Data Unit Header).

[0070] The geometric information decoding unit 2020 may be configured to decode the following syntax in addition when multi-level Trisoup is permitted, i.e., when the value of trisoup_multilevel_enabled_flag is "1".

[0071] When GSH2012 allows multi-level Trisoup, it may include syntax (log2_trisoup_max_node_size_minus2) to specify the maximum Trisoup node size.

[0072] The syntax may also be expressed as the value obtained by converting the actual maximum Trisoup node size to a base-2 logarithm. Furthermore, the syntax may also be expressed as the value obtained by converting the actual maximum Trisoup node size to a base-2 logarithm and then subtracting 2.

[0073] When GSH2012 allows multi-level Trisoup, it may include syntax (log2_trisoup_min_node_size_minus2) that specifies a minimum Trisoup node size.

[0074] The syntax may also be expressed as the base-2 logarithmic scale of the actual minimum Trisoup node size. 2 Alternatively, it may be expressed as the value obtained by subtracting 2 after converting to a base-2 logarithm.

[0075] Furthermore, the value of the syntax may be constrained to be always greater than or equal to 0 and less than or equal to log2_trisoup_max_node_size_minus2.

[0076] Alternatively, as shown in Figure 5, trisoup_depth may be defined as trisoup_depth = log2_trisoup_max_node_size_minus2 - log2_trisoup_min_node_size_minus2 + 1.

[0077] Alternatively, instead of directly decoding the minimum and maximum Trisoup node sizes, the Depth values ​​corresponding to the maximum and minimum Trisoup node sizes in the Octree processing described later may be decoded.

[0078] For example, if the maximum depth (the depth at which all nodes are 1x1x1 in size) is 10, the minimum Trisoup node size is 4 (=2 2 ), Maximum Trisoup node size is 16 (=2 4 If you want to do this, you can decrypt 8 as the Depth value corresponding to the minimum Trisoup node size and 6 as the Depth value corresponding to the maximum Trisoup node size.

[0079] When GSH2012 allows multi-level Trisoup, it may include syntax (log2_trisoup_ctu_size_minus2) that specifies the size of the node that decrypts the Trisoup node size (hereinafter referred to as CTU).

[0080] Such syntax may also be expressed as a value converted to a base-2 logarithm.

[0081] Furthermore, this syntax sets the minimum actual Trisoup node size to 4 (=2 2 Alternatively, it may be expressed as the value obtained by subtracting 2 after converting to a base-2 logarithm.

[0082] Furthermore, the values ​​of such syntax may be constrained to always be greater than or equal to the maximum Trisoup node size.

[0083] Alternatively, the value of such syntax may be expressed as the value obtained by subtracting the value obtained by converting the maximum Trisoup node size to a base-2 logarithm from the value obtained by converting the CTU size to a base-2 logarithm.

[0084] The geometric information decoding unit 2020 may be configured to decode the following syntax in addition when multi-level Trisoup is not permitted, i.e., when the value of trisoup_multilevel_enabled_flag is "0".

[0085] As described above, the geometric information decoding unit 2020 in this embodiment may be configured to decode the maximum Trisoup node size, which is the maximum node size to which the above-mentioned Trisoup is applied, and the minimum Trisoup node size, which is the minimum node size to which the Trisoup is applied.

[0086] Furthermore, as described above, the geometric information decoding unit 2020 in this embodiment may be configured to decode a predetermined size (CTU size) as a value greater than or equal to the maximum Trisoup node size.

[0087] This configuration allows for the selection of node sizes between the maximum and minimum Trisoup node sizes for each predetermined area (CTU).

[0088] Furthermore, as described above, the geometric information decoding unit 2020 in this embodiment may be configured to set the maximum Trisoup node size to a predetermined value.

[0089] This configuration eliminates the need for CTU-sized decoding, thereby reducing the amount of data encoded and the processing load.

[0090] GSH2012 may include syntax (log2_trisoup_node_sizeTrisoup_node_size_minus2) to specify the Trisoup node size when Trisoup is used but multiple levels of Trisoup are not permitted.

[0091] Such syntax may also be expressed as a value obtained by converting the actual Trisoup node size to a base-2 logarithmic scale.

[0092] Furthermore, such syntax may also be expressed as the value obtained by converting the actual Trisoup node size to a base-2 logarithm and then subtracting 2.

[0093] Alternatively, as shown in Figure 5, trisoup_depth may be defined as trisoup_depth=1.

[0094] GSH2012 may include syntax (trisoup_sampling_value_minus1) to control the subsampling interval of reconstruction points when using Trisoup.

[0095] Alternatively, instead of using the syntax shown, a threshold value (trisoup_sampling_threshold) indicating the maximum number of points after subsampling may be decoded, as shown in Figure 6. Specific subsampling methods based on these syntax values ​​will be described later.

[0096] GSH2012 may include syntax (trisoup_vertex_number_bits) that specifies the precision (number of bits) of the vertex position in Trisoup, as described later. For example, if the value of this syntax is 2, it means that the vertex position is 2 bits, that is, it can take on four different values: 0, 1, 2, and 3.

[0097] Here, trisoup_vertex_number_bits may always transmit only one value regardless of the value of trisoup_depth, as described above, or the number of bits to decode may change depending on the value of trisoup_depth.

[0098] For example, each trisoup_vertex_number_bits corresponding to each trisoup_depth may be decoded. In other words, the same number of trisoup_vertex_number_bits as trisoup_depth may be decoded. Specifically, for example, if trisoup_depth is 2, two different values ​​of trisoup_vertex_number_bits may be transmitted.

[0099] GSH2012 may include a flag (trisoup_centroid_vertex_residual_flag) indicating whether or not to decode the centroid residuals of the Trisoup vertices, as described later. For example, the flag may be defined such that a value of 1 means to decode the centroid residuals, and a value of 0 means not to decode the centroid residuals.

[0100] Here, trisoup_centroid_vertex_residual_flag may always transmit only one value regardless of the value of trisoup_depth, as described above, or it may change the number of values ​​to decode depending on the value of trisoup_depth.

[0101] For example, each trisoup_centroid_vertex_residual_flag corresponding to each trisoup_depth may be decoded. In other words, the same number of trisoup_centroid_vertex_residual_flags as trisoup_depth may be decoded. Specifically, for example, if trisoup_depth is 2, two different values ​​of trisoup_centroid_vertex_residual_flag may be transmitted.

[0102] When GSH2012 uses Trisoup and allows Trisoup at multiple levels, it may include a flag (unique_segments_exist_flag[i]) indicating whether a unique segment exists in the target hierarchy for each hierarchy i (i=0,...,trisoup_depth-1).

[0103] For example, if the value of unique_segments_exist_flag[i] is "1", it means that there is at least one unique segment in hierarchy i. Conversely, if the value of unique_segments_exist_flag[i] is "0", it means that there are no unique segments in hierarchy i.

[0104] GSH2012 may also include syntax indicating the number of bits in the syntax indicating the number of unique segments in the target hierarchy (num_unique_segments_bits_minus1[i]) and syntax indicating the number of unique segments in the target hierarchy (num_unique_segments_minus1[i]) if a unique segment exists in the target hierarchy for each hierarchy i (i=0,...,trisoup_depth-1), i.e., if the value of unique_segments_exist_flag[i] is "1".

[0105] Here, for both num_unique_segments_bits_minus1[i] and num_unique_segments_minus1[i], we may encode the syntax value as the original value minus "1" for each.

[0106] (Tree Synthesis Department 2020) The processing of the tree synthesis unit 2020 will be explained below using Figure 7. Figure 6 is a flowchart showing an example of the processing in the tree synthesis unit 2020. The following explanation will describe an example of synthesizing a tree using "Octree".

[0107] In step S601, the tree synthesis unit 2020 checks whether all depth processing has been completed. The depth count may be included as control data in the bitstream transmitted from the point cloud encoding device 100 to the point cloud decoding device 200.

[0108] The tree synthesis unit 2020 calculates the node size of the target Depth. In the case of "Octree", the node size of the first Depth may be defined as "2 to the power of the number of Depths". That is, if the number of Depths is N, the node size of the first Depth may be defined as 2 to the power of N.

[0109] Furthermore, the node size for the second and subsequent Depths may be defined by decreasing the number of N by one each time. That is, the node size for the second Depth may be defined as "2 to the power of (N-1)", the node size for the third Depth may be defined as "2 to the power of (N-2)", and so on.

[0110] Alternatively, since node size is always defined as a power of 2, one can simply consider the value of the exponent (N, N-1, N-2, etc.) as the node size. In the following explanation, node size will refer to the value of the exponent of the length of one side of the node.

[0111] For simplicity, the following explanation will use the case where the node shape is a cube, that is, where all sides of the node are of equal length.

[0112] When using QtBt, that is, when the node shape is a rectangular prism and the length of each side of the node differs along the axial direction (x, y, z), the node size can be considered to be the length of the shortest side in the three directions. Similarly, the node size can be considered to be the length of the longest side in the three directions.

[0113] Here, if the flag controlling whether or not to use Trisoup (trisoup_enabled_flag) indicates that Trisoup should be used, that is, when the value of trisoup_enabled_flag is "1", the tree synthesis unit 2020 may change the number of depths to process based on the value of the syntax that defines the minimum size of the Trisoup node (log2_trisoup_min_node_size_minus2) or the syntax that defines the size of the Trisoup node (log2_Trisoup_node_size_minus2). In such a case, for example, it may be defined as follows:

[0114] Processing Depth = Total Depth - (Smallest) Trisoup Node Size Here, the minimum Trisoup node size can be defined, for example, as (log2_trisoup_min_node_size_minus2+2). Similarly, the Trisoup node size can be defined as (log2_Trisoup_node_size_minus2+2).

[0115] In this case, if all processing depths are completed, the tree synthesis unit 2020 proceeds to step S609; otherwise, the tree synthesis unit 2020 proceeds to step S602.

[0116] In other words, if (processing depth - n) = 0, the tree synthesis unit 2020 proceeds to step S609, and if (processing depth - n) > 0, the tree synthesis unit 2020 proceeds to step S602.

[0117] Alternatively, the tree synthesis unit 2020 may determine that Trisoup should be applied to all nodes with a node size (N - number of processing depths) when proceeding to step S609.

[0118] In step S602, the tree synthesis unit 2020 determines whether or not it is necessary to decode the Trisoup_node_size, which will be described later, at the target Depth.

[0119] For example, if "Trisoup at multiple levels is permitted (value of trisoup_multilevel_enabled_flag is "1")" and "the node size (Nn) of the target Depth is the CTU size", then the tree synthesis unit 2020 may determine that "decryption of Trisoup_node_size is required".

[0120] In this embodiment, the node that decodes the Trisoup node size is called a CTU (Coding Tree Unit). The name CTU is merely an example; any node that decodes the Trisoup node size can be named in any other way.

[0121] Furthermore, if the above conditions are not met, the tree synthesis unit 2020 may determine that "decryption of Trisoup_node_size is not necessary."

[0122] Here, the maximum Trisoup node size can be defined, for example, as (log2_trisoup_max_node_size_minus2+2).

[0123] Similarly, the minimum Trisoup node size can be defined, for example, as (log2_trisoup_min_node_size_minus2+2).

[0124] Once the above determination is complete, the tree synthesis unit 2020 proceeds to step S603.

[0125] In step S603, the tree synthesis unit 2020 determines whether processing of all nodes included in the target Depth has been completed.

[0126] If the tree synthesis unit 2020 determines that processing of all nodes in the target Depth has been completed, it moves to step S601 and processes the next Depth.

[0127] On the other hand, if processing of all nodes in the target Depth has not been completed, the tree synthesis unit 2020 moves to step S604.

[0128] In step S604, the tree synthesis unit 2020 checks whether or not it is necessary to decode the Trisoup_node_size determined in step S602.

[0129] Alternatively, step S602 may be omitted, and at the processing timing of step S604, the necessity of decrypting Trisoup_node_size may be determined in the same way as in step S602.

[0130] If it is determined that decryption of Trisoup_node_size is necessary, the tree synthesis unit 2020 proceeds to step S605. If it is determined that decryption of Trisoup_node_size is not necessary, the tree synthesis unit 2020 proceeds to step S606.

[0131] In step S605, the tree synthesis unit 2020 decodes Trisoup_node_size.

[0132] Trisoup_node_size indicates the size at which Trisoup should be applied to the descendant nodes obtained by recursively partitioning the CTU using Octree or QtBt.

[0133] For example, if the CTU size is 5 (=2 5 =32), and the decrypted Trisoup_node_size is 2 (=2 2 If =4), this may mean that Trisoup is applied to each node (node ​​size 2) obtained when the CTU is divided three times (=5-2) by Octree.

[0134] The node size to which the decrypted Trisoup is applied (hereinafter referred to as the Trisoup node size) is stored as additional information for that node. Note that the initial value of the Trisoup node size is 0 (=2 0 You can set it to =1 and overwrite it with the decrypted value in step S605.

[0135] After decoding Trisoup_node_size, the tree synthesis unit 2020 proceeds to step S606.

[0136] In step S606, the tree synthesis unit 2020 checks the value of the Trisoup node size stored as internal information for that node.

[0137] When applying Trisoup to a target node, that is, when the size of the node is equal to the size of the Trisoup node stored as internal information of that node, the tree synthesis unit 2020 proceeds to step S607.

[0138] If Trisoup is not applied to the target node, that is, if the size of the node and the size of the Trisoup node stored as internal information of the node are different, the tree synthesis unit 2020 proceeds to step S608.

[0139] In step S607, the tree synthesis unit 2020 stores the target node as a node to which Trisoup will be applied, i.e., a Trisoup node. No further node splitting by "Octree" will be applied to this target node. After that, the tree synthesis unit 2020 proceeds to step S603 and moves on to processing the next node.

[0140] In step S608, the tree synthesis unit 2020 decodes information called the occupancy code.

[0141] In the case of "Octree," the occpancy code is information that indicates whether or not the point to be decrypted is contained within each of the eight nodes (called child nodes) obtained by dividing the target node in half along the x, y, and z axes.

[0142] For example, the occupancy code may assign one bit of information to each child node, where a "1" bit of information defines that the child node contains a point to be decoded, and a "0" bit of information defines that the child node does not contain a point to be decoded.

[0143] When decoding such occupancy code, the tree synthesis unit 2020 may pre-estimate the probability that a point to be decoded exists in each child node, and entropy decode the bits corresponding to each child node based on that probability.

[0144] Furthermore, in step S608, the tree synthesis unit 2020 may store the position information of nodes to which Trisoup was not applied, for each Depth, in a one-dimensional array or the like, for use in S901, which will be described later.

[0145] Specifically, the tree synthesis unit 2020 may store the position information of the target node (the node before division) in step S608. For example, the tree synthesis unit 2020 may store the coordinate values ​​of the point closest to the origin among the vertices of the target node (cuboid) in a one-dimensional array for each depth, and this information may be used by the approximate surface synthesis unit 2030, which will be described later.

[0146] If the size of a rectangular prism (the length of the sides in the x, y, and z directions) is determined for each depth, then, as described above, if the coordinates and depth of the point closest to the origin within the node can be identified, the positional information of that node can be reconstructed.

[0147] Similarly, the point cloud coding device 100 may perform entropy coding.

[0148] Furthermore, in step S608, the Trisoup node size stored as internal information of the node may be stored as internal information of each child node of that node. In other words, the Trisoup node size may be inherited by the child nodes. This allows the value of the Trisoup node size decoded in step S605 to be propagated to the descendant nodes.

[0149] (Approximate surface synthesis section 2030) Below, an example of the processing of the approximate surface composite section 2030 will be explained using Figures 8 to 20.

[0150] Figure 8 is a flowchart showing an example of the processing of the approximate surface synthesis section 2030.

[0151] As shown in Figure 8, in step S701, the approximate surface synthesis unit 2030 determines whether processing at all trisoup_depths has been completed.

[0152] If processing for all trisoup_depths is complete, proceed to step S709. If processing for all trisoup_depths is not complete, proceed to step S702.

[0153] In step S702, the approximate surface synthesis unit 2030 determines whether the Trisoup node size corresponding to the trisoup_depth is equal to the maximum Trisoup node size.

[0154] If both are equal, the operation proceeds to step S704. If they are not equal, that is, if the Trisoup node size corresponding to the trisoup_depth is smaller than the maximum Trisoup node size, the operation proceeds to step S703.

[0155] In step S702, the approximate surface synthesis unit 2030 acquires and integrates the vertex positions for each node.

[0156] In step S703, the approximate surface synthesis unit 2030 generates segments corresponding to the "interpolated vertices (vertices that have undergone interpolation)" generated in step S708, which will be described later. Figure 9 is a flowchart showing an example of the process in step S703.

[0157] Below, an example of the process in step S703 will be explained using Figure 9.

[0158] As shown in Figure 9, in step S801, the approximate surface synthesis unit 2030 determines whether processing has been completed for all "interpolated vertices" generated in step S708, which will be described later.

[0159] If completed, this operation proceeds to step S805 and terminates. If not completed, this operation proceeds to step S802 to process the next "interpolated vertex".

[0160] In step S802, the approximate surface synthesis unit 2030 determines which direction (x, y, or z direction) the "interpolated vertex" lies on in the segment.

[0161] For example, as will be described later, if the vertices of a segment and Trisoup are located at a position shifted by 0.5 from their integer coordinate positions, then in the x-direction segment, only the x-coordinate of the "interpolated vertex" will be an integer value (x.0), while the y-coordinate and z-coordinate will be decimal values ​​(x.5).

[0162] Similarly, in the y-direction segment, only the y-coordinate is an integer value, and in the z-direction segment, only the z-coordinate is an integer value. Therefore, the approximate surface synthesis unit 2030 can determine the direction of the segment in which the "interpolated vertex" should be located by checking which axial coordinate is an integer.

[0163] Furthermore, if the coordinate values ​​are stored in integer format, for example, if the true coordinate value is doubled and then stored as an integer to represent 0.5, the approximate surface synthesis unit 2030 can determine whether the value is an integer or a decimal (x.5) by checking whether the least significant bit of the coordinate value in each axis direction is 1.

[0164] The approximate surface synthesis unit 2030 saves the results of this determination and proceeds to the next step S803.

[0165] In step S803, the approximate surface compositing unit 2030 determines whether the "interpolated vertex" lies on an edge of the current Trisoup node size. For example, the approximate surface compositing unit 2030 can make such a determination in the following way. (1) Firstly, the approximate surface synthesis unit 2030 quantizes the x, y, and z coordinate values ​​of the "interpolated vertices" into integers.

[0166] Specifically, the approximate surface synthesis unit 2030 adds 0.5 to each coordinate value and then truncates the decimal part.

[0167] Here, if the approximate surface synthesis unit 2030 stores the coordinate values ​​as integer values ​​that are twice the true values, it adds 1, then divides by 2 and truncates the decimal part.

[0168] Alternatively, if the approximate surface synthesis unit 2030 stores the coordinate values ​​as integer values ​​that are twice the true values, it adds 1 and then shifts them one bit to the right.

[0169] Although this explanation describes the case where all axes of the x, y, and z directions are converted to integers, the approximate surface synthesis unit 2030 may also be configured to convert only the axes other than the "direction" determined in step S802 to integers.

[0170] For example, in the approximate surface synthesis section 2030, if the "direction" mentioned above is the x-axis, only the y-coordinate and z-coordinate need to be integerized. (2) Secondly, the approximate surface synthesis unit 2030 checks whether the integer coordinate values ​​of the axes other than the "direction" determined in step S802 are multiples of the Trisoup node size.

[0171] For example, in the approximate surface synthesis section 2030, the "direction" described above is the x-axis direction, and the value of the Trisoup node size is 8 (=2 3 In this case, check whether the integerized y-coordinate and z-coordinate are both multiples of 8.

[0172] In the process described in (2) above, if the integer coordinate values ​​of all axes other than "direction" are multiples of the Trisoup node size, the approximate surface synthesis unit 2030 determines that the "interpolated vertex" lies on an edge of the current Trisoup node size and proceeds to step S804.

[0173] Otherwise, that is, if at least one axial coordinate value is not a multiple of the Trisoup node size, the approximate surface synthesis unit 2030 determines that the "interpolated vertex" does not exist on an edge of the current Trisoup node size, and proceeds to step S801 to process the next "interpolated vertex".

[0174] In step S804, the approximate surface synthesis unit 2030 generates segments corresponding to the "interpolated vertices" using the coordinates of the "interpolated vertices" that were integerized in step S803.

[0175] Here, a segment may consist of three elements: the coordinates of the starting point (x, y, z), the coordinates of the ending point (x, y, z), and the vertex position on the segment (0 to Trisoup node size - 1).

[0176] The starting point mentioned above can be obtained, for example, by quantizing the coordinate values ​​of the integerized "interpolated vertices" by the Trisoup node size.

[0177] Specifically, for example, if we take the base-2 logarithm of the Trisoup node size and define it as TrisoupNodeSizeLog2, then we can derive it by performing the following operations on the coordinate values ​​of the "interpolated vertices".

[0178] Quantized coordinate value = (coordinate value >> TrisoupNodeSizeLog2) <TrisoupNodeSizeLog2 Here, >> represents a right bit shift, and << represents a left bit shift.

[0179] Furthermore, if the process proceeds to step S804, it is clear that the coordinate values ​​of axes other than the "direction" determined in step S802 are already multiples of the Trisoup node size. Therefore, in step S804, the approximate surface synthesis unit 2030 may perform the above quantization process only on the coordinate values ​​of the axis corresponding to the "direction" determined in step S802.

[0180] Next, the coordinates of the endpoint can be calculated by adding the value of the Trisoup node size to the coordinate value of the axis corresponding to the "direction" determined in step S802, relative to the coordinates of the starting point.

[0181] Finally, the vertex position on the segment can be derived by subtracting the coordinate value of the starting point on the axis corresponding to the "direction" from the integer coordinate value of the "interpolated vertex" on the axis corresponding to the "direction" determined in step S802.

[0182] After performing the above steps, the process proceeds to step S801 and then to the next step, "interpolated vertices".

[0183] As described above, the approximate surface synthesis unit 2030 generates segments from the "interpolated vertices" in step S703 of Figure 8, and then proceeds to step S704.

[0184] In step S704, the approximate surface synthesis unit 2030 collects adjacent information to be used for decoding the vertices in the next step S705.

[0185] Figure 10 is a flowchart illustrating an example of the process in step S704. The following explanation will use Figure 10 to describe an example of the process in step S704.

[0186] As shown in Figure 10, in step S901, the approximate surface synthesis unit 2030 generates mask information. Figure 11 shows a specific example of the method for generating mask information.

[0187] When generating mask information, as shown in Figure 11, the approximate surface synthesis unit 2030 determines in step S1001 whether processing has been completed for all Trisoup nodes within the given Trisoup node size.

[0188] If completed, this operation proceeds to step S1003; otherwise, this operation proceeds to step S1002.

[0189] In step S1002, the approximate surface synthesis unit 2030 generates 36 segments and mask values ​​corresponding to each segment for the Trisoup node.

[0190] Figure 12 shows an example of segment and mask value generation. The shaded area in Figure 12 shows an example of the Trisoup node, each line segment shows an example of a segment, and the numbers written on the segments show an example of a mask value.

[0191] Note that it is difficult to provide examples of mask values ​​for all segments, so examples of mask values ​​are provided only for some segments. However, in reality, mask values ​​should be set for all segments as follows.

[0192] First, the approximate surface synthesis unit 2030 generates four segments each for each of the 12 sides of the Trisoup node in the x, y, and z directions, at adjacent positions with larger coordinate values ​​than the node, and at adjacent positions with smaller coordinate values, thereby generating a total of 36 segments as shown in Figure 12.

[0193] Here, each segment holds information about its start and end points, but does not retain information about vertex positions, as there are no vertices.

[0194] Next, we will explain examples of setting mask values ​​for each segment.

[0195] The approximate surface synthesis unit 2030 sets a mask value for each segment corresponding to the 12 edges of the Trisoup node, such that only one of the lower four bits of the mask value is 1, and the others are 0.

[0196] For example, as shown in Figure 12, the approximate surface synthesis section 2030 may have mask values ​​of 1, 2, 4, and 8 set for the segments in the x-axis direction.

[0197] Similarly, the approximate surface composite section 2030 may have mask values ​​of 1+(1<<13), 2+(1<<13), 4+(1<<13), and 8+(1<<13) set for the segments in the y-axis direction.

[0198] Similarly, the approximate surface composite section 2030 may have mask values ​​of 1+(1<<14), 2+(1<<14), 4+(1<<14), and 8+(1<<14) set for the z-axis segment.

[0199] When configured in this way, if any of the first to fourth bits of the mask value are 1, it can be determined that the node corresponding to that segment is a Trisoup node.

[0200] Next, the approximate surface synthesis unit 2030 may set mask values ​​of 16, 32, 64, and 128, respectively, for segments corresponding to adjacent nodes in the x, y, and z directions where the coordinate values ​​increase from the Trisoup node (a total of 12 segments, four in each of the x, y, and z directions), as shown in Figure 12.

[0201] With this setting, if any of the 5th to 8th bits of the mask value are 1, it can be determined that the adjacent node with a smaller coordinate value for that segment is a Trisoup node.

[0202] Next, the approximate surface synthesis unit 2030 may set mask values ​​of 256, 512, 1024, and 2048, respectively, for segments corresponding to adjacent nodes in the x, y, and z directions where the coordinate values ​​are smaller than those of the Trisoup node (a total of 12 segments, four in each of the x, y, and z directions), as shown in Figure 12.

[0203] With this setting, if any of the 9th to 12th bits of the mask value are 1, it can be determined that the adjacent node with the larger coordinate value for that segment is a Trisoup node.

[0204] As described above, after the approximate surface synthesis unit 2030 generates segments and their corresponding mask values, it proceeds to step S1001 and moves on to processing the next Trisoup node.

[0205] Here, in step S1001, nodes that were saved in step S608 above and to which Trisoup was not applied in that Depth may be added as targets for processing in this step.

[0206] For "nodes to which Trisoup was not applied at the given Depth," no Trisoup vertices or anything similar are generated on the node itself because Trisoup is not applied at that Depth. However, this is an area where Trisoup should be applied at a Depth greater than that Depth. Therefore, when generating a mask for "nodes to which Trisoup was not applied at the given Depth" in step S1002, a mask where "any of the 1st to 4th bits of the mask value is 1" is not set. Only a mask where "any of the 5th to 8th bits of the mask value is 1" and a mask where "any of the 9th to 12th bits of the mask value is 1" are set.

[0207] In step S1003, the approximate surface synthesis unit 2030 determines whether it has completed processing all segments corresponding to the "interpolated vertices" generated in step S703.

[0208] If completed, this operation proceeds to step S1005 and terminates the mask generation process in step S901. If not completed, this operation proceeds to step S1004.

[0209] In step S1004, the approximate surface synthesis unit 2030 generates a mask value for the segment corresponding to the "interpolated vertex" that indicates the existence of the "interpolated vertex" in that segment.

[0210] The approximate surface synthesis unit 2030 may set this mask value to a value of, for example, 1 << 15. If such a value is set, it can be determined that an "interpolated vertex" exists in the segment if the 16th bit of the mask value is 1.

[0211] After generating the mask value, the approximate surface synthesis unit 2030 proceeds to step S1003 and then proceeds to processing the next segment.

[0212] Furthermore, the mask values ​​generated in steps S1002 and S1004 are both powers of 2. This allows for easy synthesis of mask values ​​by performing a bitwise OR operation when combining the mask values ​​of segments located at the same position in step S905, which will be described later.

[0213] Furthermore, depending on whether each bit of the resulting composite mask value is 1 or 0, information about the segment (whether it is a Trisoup node, whether there are adjacent Trisoup nodes, whether there are "interpolated vertices" in the segment, etc.) can be obtained, as described above.

[0214] After generating the mask information as described above, the approximate surface synthesis unit 2030 proceeds to step S902.

[0215] In step S902, the approximate surface synthesis unit 2030 combines all the segments generated in step S901 and the segments corresponding to the "interpolated vertices" and then sorts them.

[0216] The approximate surface synthesis unit 2030 sorts, for example, each segment based on its start and end point coordinates. By performing this process, segments with the same start and end points can be rearranged to be consecutive.

[0217] After the sorting is completed, the approximate surface synthesis unit 2030 proceeds to step S903.

[0218] In step S903, the approximate surface synthesis unit 2030 determines whether or not processing of all segments has been completed for the segments sorted in step S902.

[0219] If completed, this operation proceeds to step S909 and terminates the adjacent information collection process in step S704. If not completed, this operation proceeds to step S904.

[0220] In step S904, the approximate surface synthesis unit 2030 determines whether the segment is in the same position as the segment processed immediately before.

[0221] Specifically, the approximate surface synthesis unit 2030 determines, for example, whether both the coordinates of the start point and the coordinates of the end point of the segment are the same as those of the segment processed immediately before.

[0222] If the current segment is in the same position as the previously processed segment, the operation proceeds to step S905. If the current segment is not in the same position as the previously processed segment, the operation proceeds to step S906.

[0223] In step S905, the approximate surface synthesis unit 2030 integrates the mask value corresponding to the segment generated in step S901 with the mask values ​​of each segment at the same location that were processed immediately prior to that.

[0224] For example, if the approximate surface synthesis unit 2030 has mask values ​​that are powers of 2 corresponding to each segment, as explained in step S901, the masks can be integrated by taking the logical OR of the mask values ​​of each segment.

[0225] After integrating the mask, the approximate surface synthesis unit 2030 proceeds to step S907.

[0226] In step S906, the approximate surface synthesis unit 2030 saves the segment processed immediately before the segment in question as a unique segment if it satisfies predetermined condition A.

[0227] For example, condition A may be defined as the case where the segment processed immediately before is the segment corresponding to the Trisoup node.

[0228] For example, if at least one of the lower four bits of the mask value integrated in step S905 is 1, then it can be determined that the segment processed immediately before is the segment corresponding to the Trisoup node.

[0229] The approximate surface synthesis unit 2030 may store information about a unique segment, such as the coordinates of the starting point, the coordinates of the ending point, the interpolation point information described in step S907 below, the adjacent segment information described in step S908 below, and the mask value integrated in step S905. This information is used in the vertex decoding process in step S705.

[0230] The approximate surface synthesis unit 2030 stores information about the unique segment and then initializes various types of information (interpolation point information, adjacent segment information, integrated mask value, etc.).

[0231] Furthermore, if the predetermined conditions are not met, the approximate surface synthesis unit 2030 will only perform initialization. After the above processes are completed, this operation proceeds to step S907.

[0232] In step S907, the approximate surface synthesis unit 2030 stores information about the interpolation point if the predetermined condition B is met.

[0233] For example, predetermined condition B may be that the mask value of the segment indicates that there are "interpolated vertices" in the segment. Specifically, for example, predetermined condition B may be that the 16th bit of the mask value is 1.

[0234] The approximate surface synthesis unit 2030 stores the coordinates of the "interpolated vertices" if the predetermined condition B is met. Specifically, the approximate surface synthesis unit 2030 stores the values ​​of the "vertex positions on the segment" generated in step S804. The values ​​stored here are then stored as information about the unique segment in step S906.

[0235] If the segment is not determined to be a unique segment in step S906 (i.e., the predetermined condition A in step S906 is not met), the information regarding the interpolation point of the segment stored in step S907 is discarded.

[0236] The approximate surface synthesis unit 2030 may also prepare a separate array to store the values ​​of "vertex positions on the segment," and in this step, it may store an index value to identify the value corresponding to the segment in that array, instead of the coordinate value itself.

[0237] After the above processes are completed, this operation proceeds to step S908.

[0238] In step S908, the approximate surface synthesis unit 2030 stores information about adjacent segments.

[0239] Specifically, for example, the approximate surface synthesis unit 2030 stores the starting coordinates of the segment, and information for identifying segments with the same starting or ending coordinates (e.g., segment index).

[0240] For example, the starting coordinates of the segment in question, and segments with the same starting or ending coordinates, specifically include one segment in the same axial direction as the segment in question whose ending coordinates are the same as the starting coordinates of the segment in question, and four segments in the axial direction orthogonal to the segment in question. The information stored here is saved as information about unique segments in step S906.

[0241] If the segment in question is not determined to be a unique segment in step S906 (i.e., the predetermined condition A in step S906 is not met), the information on adjacent segments of that segment generated in step S908 is discarded.

[0242] After the above process is completed, this operation proceeds to step S903 and then to the processing of the next segment.

[0243] As described above, after collecting adjacent information in step S704, the approximate surface synthesis unit 2030 proceeds to step S705.

[0244] In step S705, the approximate surface synthesis unit 2030 decodes the vertices of the Trisoup at the trisoup_depth.

[0245] Figure 13 is a flowchart showing an example of a vertex decoding method in step S705. An example of the process in step S705 will be explained below using Figure 13.

[0246] As shown in Figure 13, in step S1201, the approximate surface synthesis unit 2030 determines whether processing has been completed for all unique segments generated in step S703.

[0247] If processing is completed for all unique segments, this operation proceeds to step S1207 and terminates the processing in step S703. If processing is not completed for all unique segments, this operation proceeds to step S1202.

[0248] In step S1202, the approximate surface synthesis unit 2030 determines whether or not an "interpolated vertex" exists in the unique segment.

[0249] Whether or not "interpolated vertices" exist can be determined based on the integrated mask value saved in step S906. If "interpolated vertices" exist, the process proceeds to step S1203. If "interpolated vertices" do not exist, the process proceeds to step S1204.

[0250] In step S1203, the approximate surface synthesis unit 2030 saves the "vertex position on the segment" saved in step S906 as the vertex position in the unique segment.

[0251] In other words, the approximate surface synthesis unit 2030 uses the "vertex position on the segment" saved in step S906 as the decoded value of the vertex position in that unique segment.

[0252] Furthermore, the approximate surface synthesis unit 2030 sets the decoded value of "presence or absence of vertices" in the unique segment to "vertices present".

[0253] In other words, step S1203 is a process in which, instead of decoding the information indicating the vertex position and presence or absence of a vertex from the bitstream, the information indicating the vertex position and presence or absence of a vertex is implicitly determined using interpolated values.

[0254] After the above process is completed, this operation proceeds to step S1201 and then proceeds to processing the next unique segment.

[0255] In step S1204, the approximate surface synthesis unit 2030 decodes information from the bitstream indicating whether or not a vertex exists in the unique segment.

[0256] Information indicating the presence or absence of a vertex is a 1-bit flag, which may be entropy-encoded in the point cloud coding device 100. The approximate surface synthesis unit 2030 may prepare multiple probabilistic models for entropy coding (coding device) / decoding (decoding device) and select one according to the context. The context may be set, for example, based on the mask value saved and integrated in step S906.

[0257] The approximate surface synthesis unit 2030 decodes the information indicating the presence or absence of vertices, and then proceeds to step S1205.

[0258] In step S1205, the approximate surface synthesis unit 2030 determines whether or not a vertex exists in the unique segment.

[0259] Based on the information indicating the presence or absence of vertices decoded in step S1204, the approximate surface synthesis unit 2030 proceeds to step S1206 if vertices exist. If no such vertices exist, the approximate surface synthesis unit 2030 proceeds to step S1201 and then proceeds to processing the next unique segment.

[0260] In step S1206, the approximate surface synthesis unit 2030 decodes the "vertex position on the segment" for the unique segment in which a vertex is determined to exist. Here, the vertex position may be entropy encoded. For entropy encoding (encoding device) / decoding (decoding device), the approximate surface synthesis unit 2030 may prepare multiple probabilistic models and select one according to the context. The context may be set, for example, based on the information of adjacent segments saved in step S906.

[0261] Here, the vertex positions may be decoded with the bit precision specified by the syntax (trisoup_vertex_number_bits) used to specify the precision (number of bits) of the vertex positions in Trisoup, as described above.

[0262] For example, if trisoup_vertex_number_bits is 2, it may be decoded to take on four different values: 0, 1, 2, and 3. Furthermore, if trisoup_vertex_number_bits is decoded for each depth, it may be decoded with a bit precision corresponding to that depth.

[0263] Alternatively, the bit precision specified by trisoup_vertex_number_bits may be used only for the Depth corresponding to the smallest Trisoup node size (the largest Depth), and thereafter, the bit precision may be increased by 1 each time the node size increases by 1 (the Depth decreases by 1).

[0264] For example, if there are two node sizes (minimum and maximum), the bit precision for decoding the vertex position may be the value of trisoup_vertex_number_bits for the Depth corresponding to the minimum Trisoup node size, and the bit precision for decoding the vertex position may be the value of trisoup_vertex_number_bits plus 1 for the Depth corresponding to the maximum Trisoup node size.

[0265] After decoding the vertex positions, the approximate surface synthesis unit 2030 proceeds to step S1201 and then to processing the next unique segment.

[0266] As described above, the approximate surface synthesis unit 2030 decodes the vertices in step S705 and then proceeds to step S706.

[0267] In step S706, the approximate surface synthesis unit 2030 generates a reconstructed point cloud based on the vertices decoded in step S705. Figure 14 is a flowchart showing an example of a method for generating a reconstructed point cloud. An example of the process in step S706 will be described below using Figure 14.

[0268] As shown in Figure 14, in step S1301, the approximate surface synthesis unit 2030 determines whether processing has been completed for all Trisoup nodes in the trisoup_depth.

[0269] If processing is complete for all Trisoup nodes, the process proceeds to step S1306 and terminates. If processing is not complete for all Trisoup nodes, the process proceeds to step S1302.

[0270] In step S1302, first, the approximate surface synthesis unit 2030 identifies unique segments corresponding to each side (segment) of the Trisoup node. For example, the approximate surface synthesis unit 2030 can identify by searching for a unique segment among the unique segments processed in step S705 where the start coordinates and end coordinates of each side (segment) of the Trisoup node are the same unique segment.

[0271] Second, if vertices exist in the identified unique segment, the approximate surface synthesis unit 2030 adds the vertices to the reconstructed point cloud in the following procedure. (1) The approximate surface synthesis unit 2030 shifts the position of the segment by -0.5. Specifically, for each of the start coordinates and end coordinates of the segment, the approximate surface synthesis unit 2030 subtracts 0.5 from the coordinate value other than the coordinate axis along the direction of the segment.

[0272] For example, if the start coordinates of the segment are (x, y, z) = (12, 100, 32), the node size is 4, and the direction of the segment is the x-axis, the end coordinates are obtained by adding the node size to the x-coordinate of the start point, resulting in (16, 100, 32).

[0273] In contrast, for coordinates other than the x-axis, that is, in this example, the y-coordinate and z-coordinate are each subtracted by 0.5. Thus, the start coordinates and end coordinates become (12, 99.5, 31.5) and (16, 99.5, 31.5), respectively. (2) The approximate surface synthesis unit 2030 adds the "vertex position on the segment" of the segment to the above start coordinates. For example, in the above example, if the "vertex position on the segment" is 2, it becomes (14, 99.5, 31.5). (3) The approximate surface synthesis unit 2030 adds 0.5 to the coordinate values other than the coordinate along the axis in the segment direction for the coordinates obtained in (2) above. For example, in the above example, it becomes (14, 100, 32). This value can also obtain the same result by adding the vertex position 2 on the unique segment to the starting point coordinates (12, 100, 32) of the unique segment before executing the procedure in (1) above. Therefore, the approximate surface synthesis unit 2030 may calculate in this way. (4) When the coordinates calculated in (3) above exist inside the Trisoup node, the approximate surface synthesis unit 2030 generates a point at the coordinates calculated in (3) above and adds it to the reconstructed point cloud.

[0274] Here, the inside of the Trisoup node means a point where the values of the x, y, and z coordinates are respectively within the range of the starting point of the Trisoup node (the point where each of the x coordinate, y coordinate, and z coordinate is the smallest) + Trisoup node size - 1.

[0275] For example, when the starting point coordinates of the Trisoup node are (x, y, z) = (12, 100, 32) and the Trisoup node size is 4, the points existing in the range of (12 to 15, 100 to 103, 32 to 35) are regarded as the inside of the Trisoup node.

[0276] In this case, if the coordinates calculated in (3) above are (14, 100, 32), this coordinate is determined to be inside the Trisoup node, and the approximate surface synthesis unit 2030 adds the point of this coordinate to the reconstructed point cloud. FIG. 16A shows such a result in the case of projection onto the x - y plane.

[0277] On the other hand, for example, if the starting coordinates of the Trisoup node are (x, y, z) = (12, 96, 32), then points in the range of (12~15, 96~99, 32~35) are inside the Trisoup node. In this case, (14, 100, 32) is determined not to be inside the Trisoup node, and the coordinates calculated in (3) above are not added to the reconstructed point cloud, and the process ends. Figure 16B shows an example of such a case.

[0278] After completing the above process, this operation proceeds to step S1303.

[0279] In step S1303, the approximate surface synthesis unit 2030 calculates the initial coordinates of the centroid from the vertex corresponding to the Trisoup node.

[0280] For example, the initial coordinates of the centroid can be calculated by averaging the x, y, and z components of all vertex positions of the Trisoup node. Note that if the Trisoup node has three or fewer vertices, the centroid calculation may be omitted.

[0281] After this process is completed, the operation proceeds to step S1304.

[0282] In step S1304, the approximate surface synthesis unit 2030 performs vertex sorting and projection plane determination processing. Specifically, for example, the approximate surface synthesis unit 2030 can sort the vertices and determine the projection plane in the following procedure. (1) The approximate surface synthesis unit 2030 projects the vertices onto the xy plane. This process is equivalent to extracting the x and y coordinate values ​​from the coordinates of each vertex.

[0283] Since vertices originally lie on the edges of a node, in the projected plane, each vertex lies on the edge of a square or rectangle formed by projecting the node onto the plane. Because a square is a type of rectangle, in the following explanation, the shape of the projected node will be described as a rectangle. (2) The approximate surface synthesis unit 2030 sorts the points on the sides of the rectangle formed by projecting the nodes in a clockwise or counterclockwise order.

[0284] Furthermore, the approximate surface synthesis unit 2030 assigns a temporary index value (0, 1, 2, ...) to each vertex in the sorted order. (3) The approximate surface synthesis unit 2030 defines a triangle defined by the three points of two adjacent vertices and the centroid in sort order, and calculates the area of ​​this triangle. The area of ​​the triangle can be calculated, for example, by generating vectors that point from the centroid to the coordinates of the two vertices and using their cross product.

[0285] The approximate surface synthesis unit 2030 calculates the area of ​​each triangle for all combinations of adjacent vertices in sorted order (0, 1), (1, 2), ... (N, 0), and then sums them up.

[0286] Furthermore, if there are only three vertices, the approximate surface synthesis unit 2030 calculates the area of ​​the triangle formed by the three vertices without using a centroid. This area is also the area on the projected plane. (4) The approximate surface synthesis unit 2030 performs the same procedures as in (1) to (3) above for the xz plane and the yz plane, and uses the plane that maximizes the area calculated in (3) above as the projection plane, and adopts the sorting order and the provisional index assigned in (2) above as the final sorting order and index.

[0287] As described above, the approximate surface synthesis unit 2030 proceeds to step S1305 after sorting the vertices and determining the projection plane.

[0288] In step S1305, the approximate surface synthesis unit 2030 generates a reconstructed point cloud using the triangles generated in step S1304 and ray tracing.

[0289] Specifically, the approximate surface synthesis unit 2030 generates a reconstructed point cloud using the following procedure. (1) First, the approximate surface synthesis unit 2030 generates a triangle based on the Index and centroid determined in step S1304, similar to step S1304. However, while in step S1304 the approximate surface synthesis unit 2030 generated a triangle on a two-dimensional plane, in step S1305 it generates a triangle in three-dimensional space using all of the x, y, and z coordinates of each vertex. (2) Second, the approximate surface synthesis unit 2030 defines the normal vector of the projection plane (if the projection plane is the x - y plane, the vector in the z direction) and places it on the projection plane.

[0290] For example, the approximate surface synthesis unit 2030 sets the initial position to the point closest to the origin of the node shape (i.e., rectangle) on the projection plane. (3) Third, as the norm of the normal vector is increased, the approximate surface synthesis unit 2030 determines whether it intersects the triangle generated in (1) above. If it intersects, it calculates the coordinates.

[0291] For example, the approximate surface synthesis unit 2030 can implement such processing by using a general ray tracing method or the like. An example of the configuration of a triangle is shown in FIG. 16A. (4) Fourth, when the coordinates at which the normal vector calculated in (3) above intersects the triangle are within the Trisoup node, the approximate surface synthesis unit 2030 generates a point at such coordinate positions and adds it to the reconstructed point group. (5) Fifth, the approximate surface synthesis unit 2030 repeats the procedures of (3) and (4) above for the case where each integer coordinate position within the node (rectangle) on the projection plane is arranged.

[0292] At this time, the interval at which the normal vector is placed may be 1 (i.e., all integer coordinate positions). Here, the interval 1 is the smallest interval that can be taken. An example of the reconstructed point group generated from the triangle in FIG. 16A is shown in FIG. 16B.

[0293] After generating reconstruction points for the Trisoup node as described above, the approximate surface synthesis unit 2030 proceeds to step S1301 and then to processing the next Trisoup node.

[0294] As described above, in step S706, the approximate surface synthesis unit 2030 generates a reconstructed point cloud and then proceeds to step S707.

[0295] In step S707, the approximate surface synthesis unit 2030 determines whether the Trisoup node size at the trisoup_depth is the minimum Trisoup node size.

[0296] If the Trisoup node size is the minimum size, the process proceeds to step S701. Otherwise, if the Trisoup node size at the given trisoup_depth is greater than the minimum size, the process proceeds to step S708.

[0297] In step S708, the approximate surface synthesis unit 2030 interpolates vertices onto the node segments at the minimum Trisoup node size, based on the vertices corresponding to the Trisoup node size at the trisoup_depth decoded in step S705.

[0298] Figure 15 is a flowchart illustrating an example of the process in step S708. The following explanation will use Figure 15 to describe an example of the process in step S708. Note that the process in Figure 15 is almost identical to that in Figure 14, and the same reference numerals are used for identical processes. Only the differences from Figure 14 will be explained below.

[0299] As shown in Figure 15, in step S1402, the approximate surface synthesis unit 2030 saves the vertices of the Trisoup node as "interpolated vertices".

[0300] Specifically, the approximate surface synthesis unit 2030 saves the coordinate values ​​after performing steps (1) and (2) described in step S1302 as "interpolated vertices".

[0301] For example, in the example of step S1302, the approximate surface synthesis unit 2030 saves the coordinate values ​​(14, 99.5, 31.5) as "interpolated vertices". This is because the segment's position is defined as being shifted by 0.5 from its integer coordinate position.

[0302] Furthermore, in order to retain this value as an integer, the approximate surface synthesis unit 2030 may store it as a value obtained by doubling the true coordinate value. In the example above, the approximate surface synthesis unit 2030 may store the value (28, 199, 63) as the "interpolated vertex".

[0303] In step S1405, the approximate surface synthesis unit 2030 changes the position where the normal vector is placed in step (2) of step S1305 to the position where a segment exists when the Trisoup node is divided by the minimum Trisoup node size.

[0304] If the intersecting coordinates calculated in the same manner as in step (3) of step S1305 lie on a segment obtained by dividing the Trisoup node by the minimum Trisoup node size, the approximate surface synthesis unit 2030 saves such coordinate values ​​as "interpolated vertices".

[0305] If the coordinate values ​​were doubled in step S1402 to obtain integer values, the approximate surface synthesis unit 2030 will also save the coordinate values ​​doubled in step S1405.

[0306] The approximate surface synthesis unit 2030 saves the "interpolated vertices" generated by the above process and uses them in step S703 of the next trisoup_depth.

[0307] Furthermore, if the trisoup_depth is greater than 0, that is, if the Trisoup node size at the trisoup_depth is not the maximum Trisoup node size, there are "interpolated vertices" generated at a larger node size. However, the approximate surface synthesis unit 2030 does not delete this set of "interpolated vertices," but instead saves it by adding the "interpolated vertices" newly generated in step S1402 to the aforementioned set.

[0308] As shown in Figure 16C, the approximate surface synthesis unit 2030 may also save only the "interpolated vertices" that exist on the surface of the Trisoup node.

[0309] For convenience, steps S706 and S708 have been described as separate processes here, but since they have many common parts, they may be performed simultaneously.

[0310] For example, in step S1302, if the conditions of step S707 are met, the approximate surface synthesis unit 2030 may also perform step S1402, and similarly, in step S1305, if the conditions of step S707 are met, step S1405 may also be performed.

[0311] In step S709, the reconstructed point cloud in the slice generated by the method described above is subsampled. Figure 17 is a flowchart of an example of the process in step S709. An example of the process in step S709 will be explained below using Figure 17.

[0312] As shown in Figure 17, in step S1701, the approximate surface synthesis unit 2030 sorts the reconstruction points included in the reconstruction point cloud. This sorting is a process of rearranging the reconstruction points in a predetermined manner.

[0313] For example, the approximate surface synthesis unit 2030 may be sorted based on the coordinate values ​​of each point. Alternatively, for example, the approximate surface synthesis unit 2030 may be sorted in ascending order of the x-coordinate of each point.

[0314] In this case, for example, if there are multiple points with the same x-coordinate, the approximate surface synthesis unit 2030 may sort them in ascending order of their y-coordinates. Furthermore, if there are multiple points with the same x-coordinate and y-coordinate, the approximate surface synthesis unit 2030 may sort them in ascending order of their z-coordinates. After performing such sorting, the approximate surface synthesis unit 2030 proceeds to step S1702.

[0315] In step S1702, the approximate surface synthesis unit 2030 determines the subsampling interval S based on the syntax for controlling the subsampling interval of the reconstruction points (trisoup_sampling_value_minus1) described in Figure 5, or the threshold value indicating the maximum number of points after subsampling (trisoup_sampling_threshold) described in Figure 6.

[0316] For example, the approximate surface synthesis unit 2030 may use S as the value of the syntax that controls the subsampling interval of the reconstruction points.

[0317] Alternatively, for example, the approximate surface synthesis unit 2030 may set the minimum value of the subsampling interval of the reconstruction points to 1, and add 1 to the value of the syntax that controls the subsampling interval of the reconstruction points to obtain S.

[0318] Next, we will explain an example of a method for determining S using a threshold that indicates the maximum number of points after subsampling.

[0319] Here, the approximate surface synthesis unit 2030 may set S to the smallest positive integer that satisfies the following relationship, where N is the number of points included in the reconstructed point cloud (before subsampling). In this case, the minimum value of S is 1.

[0320] N / S≦trisoup_sampling_threshold Alternatively, the approximate surface composite section 2030 may be configured by rearranging the following equation to set S as the smallest positive integer that satisfies the following relationship.

[0321] N≦S×trisoup_sampling_threshold As described above, the approximate surface synthesis unit 2030 determines the subsampling interval S and then proceeds to step S1703.

[0322] In step S1703, the approximate surface synthesis unit 2030 performs subsampling.

[0323] For example, the approximate surface synthesis unit 2030 may subsample by assigning an index i (i=0, 1, 2, ..., N-1) to each point in the order sorted in step S1701, keeping only the points where i%S=0, and discarding the other points. Here, % is the operator for calculating the remainder.

[0324] After the subsampling is complete, the approximate surface synthesis unit 2030 proceeds to step S1704 and terminates the process.

[0325] As described above, with the above configuration, by first decoding the point cloud at a fine density and then subsampling it, it is possible to easily generate a reconstructed point cloud with a number of points less than or equal to any desired number.

[0326] Furthermore, with this configuration, the subsampling interval S is determined based on the syntax value transmitted from the point cloud coding device 100, and subsampling is performed. This allows the point cloud coding device 100 to set the syntax value while considering the subsampling density between different slices and subjective quality, thereby offering the advantage of controlling the subjective quality of the reconstructed point cloud.

[0327] After the subsampling process in step S709 is completed, the approximate surface synthesis unit 2030 proceeds to step S710 and terminates the process.

[0328] As described above, the approximate surface synthesis unit 2030 in this embodiment may be configured to decode the node size to which Trisoup is applied to descendant nodes obtained by recursively dividing each node of a predetermined size using Octree.

[0329] Furthermore, as described above, the approximate surface synthesis unit 2030 in this embodiment may be configured to perform context-adaptive decoding using information on the presence or absence of vertices and vertex positions of spatially adjacent decoded segments when decoding the presence or absence of vertices and vertex positions of each segment constituting the Trisoup node.

[0330] Furthermore, as described above, the approximate surface synthesis unit 2030 in this embodiment may be configured to perform context-adaptive decoding using information on the presence or absence of vertices and vertex positions of segments that constitute nodes of the same size as the Trisoup node among spatially adjacent decoded segments when decoding the presence or absence of vertices and vertex positions of each segment constituting the Trisoup node.

[0331] This configuration allows for the selection of appropriate Trisoup node sizes for each local region according to the characteristics of the point cloud, while ensuring that Trisoup nodes of the same size are adjacent within each local region. This improves encoding efficiency by utilizing spatial correlation.

[0332] Although Figure 8 illustrates the case where subsampling is performed on a slice-by-slice basis, this subsampling process may also be performed on a node-by-node basis.

[0333] Refer to Figure 18 to see an example of the process when subsampling is performed for each node. Figure 18 shows the process in Figure 14 with the addition of step S709, which is the subsampling process.

[0334] As shown in Figure 18, the approximate surface synthesis unit 2030 performs the S1302 to S1305 processes for the node to generate a reconstructed point cloud for that node.

[0335] Subsequently, the approximate surface synthesis unit 2030 performs the subsampling process in step S709 on the reconstructed point cloud at the node.

[0336] Basically, the same process can be achieved by replacing the reconstructed point cloud of the slice in the explanation of the process in step S709 with the reconstructed point cloud at the node in question.

[0337] In determining the subsampling interval in step S1702, the approximate surface synthesis unit 2030 determines the subsampling interval S based on the syntax (trisoup_sampling_value_minus1) that controls the subsampling interval.

[0338] As described above, the point cloud decoding device 200 in this embodiment may be configured to include a geometric information decoding unit 2010 that decodes parameters relating to the subsampling interval, and an approximate surface synthesis unit 2030 that generates a reconstructed point cloud of the slice or node, determines the subsampling interval based on the parameters relating to the subsampling interval S described above, and subsamples the reconstructed point cloud of the slice or node based on the subsampling interval S.

[0339] This configuration allows the point cloud coding device 100 to set parameters that take into account the subjective quality of the reconstructed point cloud, and also allows the number of points in the reconstructed point cloud to be kept below an arbitrary number.

[0340] Here, "arbitrary number" refers, for example, to the upper limit of the number of reconstruction points per slice as defined in the standard, and is something that must be adhered to in order to ensure interoperability between devices that meet such standards.

[0341] Furthermore, as described above, the approximate surface synthesis unit 2030 in this embodiment may be configured to perform subsampling by sorting the above-mentioned slice or the reconstructed point cloud of the node in a predetermined manner, assigning an index to the reconstructed point cloud in the sorted order, retaining only the points for which the remainder when the index value of each point is divided by the subsampling interval is 0, and discarding the other points.

[0342] This configuration allows for spatially uniform subsampling of points from the reconstructed point cloud, thereby suppressing the degradation of subjective image quality due to subsampling.

[0343] Furthermore, as described above, the point cloud decoding device 200 in this embodiment may be configured to decode a syntax for controlling the subsampling interval of the reconstruction points as a parameter related to the subsampling interval S, and the approximate surface synthesis unit 2030 may be configured to use the value of the syntax for controlling the subsampling interval S of the reconstruction points, or the value of the syntax for controlling the subsampling interval S of the reconstruction points plus 1, as the subsampling interval S and perform subsampling.

[0344] With this configuration, the point cloud decoder 200 can omit the calculations related to the derivation of the subsampling interval S.

[0345] Furthermore, as described above, the point cloud decoding device 200 in this embodiment may be configured to decode a threshold value indicating the maximum number of points after subsampling as a parameter related to the subsampling interval S, and the approximate surface synthesis unit 2030 may be configured to perform subsampling using the smallest value of S that, when the threshold value indicating the maximum number of points after subsampling is multiplied by a positive integer S, is equal to or greater than the number of points in the reconstructed point cloud of the slice.

[0346] This configuration allows the point cloud decoder 200 to explicitly know how many points or fewer the reconstructed point cloud should contain, thus facilitating operational verification.

[0347] The above example illustrates the interpolation of vertices from larger node sizes to smaller node sizes at Trisoup node boundaries of different sizes, using Figure 8. However, the reverse is also possible: vertices from smaller node sizes can be used to replace vertices from larger node sizes.

[0348] Using Figures 19 and 20, we will explain an example of the process of replacing vertices in a larger node size with vertices in a smaller node size at the boundary of Trisoup nodes of different sizes. Note that the same signs are used for processes similar to those in Figure 8, and their explanation is omitted.

[0349] As shown in Figure 19, in step S721, the approximate surface synthesis unit 2030 interpolates the vertices at the minimum node size.

[0350] Here, the basic interpolation method is the same as that described in step S708 in Figure 8 and in Figures 15 and 16. The only difference is that in step S708, as shown in Figure 16C, interpolated vertices were generated only on the surface of the original node, whereas in step S721, as shown in Figure 20, interpolated vertices are also generated inside the original node.

[0351] In this way, the approximate surface synthesis unit 2030 generates vertices in units of the minimum Trisoup node size, regardless of the original Trisoup node size.

[0352] In step S722, the approximate surface synthesis unit 2030 performs vertex integration. Here, the inputs are the vertices in each segment of the minimum node size interpolated in step S721 and the vertices decoded in the minimum Trisoup node size.

[0353] In this case, if the node sizes differ between adjacent nodes, a single segment may contain multiple vertices interpolated from a larger node size and vertices decoded at the minimum Trisoup node size.

[0354] For example, the approximate surface synthesis unit 2030 can merge these vertices by discarding all but one point from each segment.

[0355] Specifically, for example, the approximate surface synthesis unit 2030 can integrate multiple vertices by retaining only the vertices generated from the smallest Trisoup node size.

[0356] As described above, the approximate surface synthesis unit 2030 merges the vertices of each segment into one, and then proceeds to the reconstruction point cloud generation process in step S723.

[0357] In step S723, the approximate surface synthesis unit 2030 generates a reconstructed point cloud based on the vertices integrated in step S722.

[0358] At this point, steps S721 and S722 have resulted in the vertices of the Trisoup corresponding to each segment in the minimum Trisoup node size. Therefore, by performing the process shown in Figure 14 in the minimum Trisoup node size, a reconstructed point cloud of the entire slice can be generated.

[0359] (Point cloud encoding device 100) The point cloud coding device 100 according to this embodiment will be described below with reference to Figure 21. Figure 21 is a diagram showing an example of the functional blocks of the point cloud coding device 100 according to this embodiment.

[0360] As shown in Figure 21, the point cloud coding device 100 includes a coordinate transformation unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information coding unit 1050, a geometric information reconstruction unit 1060, a color conversion unit 1070, an attribute transfer unit 1080, a RAHT unit 1090, a LoD calculation unit 1100, a lifting unit 1110, an attribute information quantization unit 1120, and an attribute information coding unit 1130.

[0361] The coordinate transformation unit 1010 is configured to perform a transformation process from the 3D coordinate system of the input point cloud to any different coordinate system. The coordinate transformation may be performed, for example, by rotating the input point cloud to transform the x, y, and z coordinates of the input point cloud into arbitrary s, t, and u coordinates. Alternatively, as one variation of the transformation, the coordinate system of the input point cloud may be used as is.

[0362] The geometric information quantization unit 1020 is configured to quantize the position information of the input point cloud after coordinate transformation and to remove points with overlapping coordinates. When the quantization step size is 1, the position information of the input point cloud and the position information after quantization coincide. In other words, when the quantization step size is 1, it is equivalent to not performing quantization.

[0363] The tree analysis unit 1030 is configured to take the positional information of the quantized point cloud as input and generate an occupancy code that indicates which node in the encoding target space a point is located at, based on the tree structure described later.

[0364] The tree analysis unit 1030 is configured to generate a tree structure in this process by recursively dividing the space to be encoded into rectangular parallelepipeds.

[0365] Here, if a point exists within a given rectangular prism, a tree structure can be generated by recursively dividing that rectangular prism into multiple rectangular prisms until the rectangular prism reaches a predetermined size. Each of these rectangular prisms is called a node. Each rectangular prism generated by dividing a node is called a child node, and the occupancy code is a representation of whether or not a point is contained within a child node, expressed as 0 or 1.

[0366] As described above, the tree analysis unit 1030 is configured to generate occupancy code by recursively dividing the nodes until they reach a predetermined size.

[0367] In this embodiment, a method called "Octree" can be used, which recursively performs octree partitioning by always treating the cuboid as a cube, and a method called "QtBt" can be used, which performs quadtree partitioning and binary tree partitioning in addition to octree partitioning.

[0368] Whether or not to use "QtBt" is transmitted to the point cloud decoder 200 as control data.

[0369] Alternatively, it may be specified to use predicative coding with an arbitrary tree structure. In this case, the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoder 200 as control data.

[0370] For example, the control data in a tree structure may be configured to be decryptable using the procedure described in Figure 6.

[0371] The approximate surface analysis unit 1040 is configured to generate approximate surface information using the tree information generated by the tree analysis unit 1030.

[0372] Approximate surface information is used, for example, when decoding 3D point cloud data of an object, in cases where the point cloud is densely distributed on the object's surface. Instead of decoding each individual point cloud, the region where the point cloud exists is approximated and represented by a small plane.

[0373] Specifically, the approximate surface analysis unit 1040 may be configured to generate approximate surface information using a method called "Trisoup," for example. Furthermore, this process can be omitted when decoding sparse point clouds acquired by Lidar or the like.

[0374] The geometric information encoding unit 1050 is configured to encode the syntax of the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040, and generate a bitstream (geometric information bitstream). Here, the bitstream may include, for example, the syntax described in Figures 4 and 5.

[0375] The encoding process is, for example, context-adaptive binary arithmetic encoding. Here, for example, the syntax includes control data (flags and parameters) to control the decoding process of the location information.

[0376] The geometric information reconstruction unit 1060 is configured to reconstruct the geometric information of each point in the point cloud data to be encoded (the coordinate system assumed by the encoding process, i.e., the position information after the coordinate transformation in the coordinate transformation unit 1010) based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040.

[0377] The color conversion unit 1070 is configured to perform color conversion if the input attribute information is color information. Color conversion is not always necessary; whether or not the color conversion process is performed is encoded as part of the control data and transmitted to the point cloud decoder 200.

[0378] The attribute transfer unit 1080 is configured to correct attribute values ​​to minimize distortion of attribute information based on the position information of the input point cloud, the position information of the point cloud after reconstruction by the geometric information reconstruction unit 1060, and the attribute information after color change by the color conversion unit 1070.

[0379] The RAHT unit 1090 is configured to take the attribute information after the attribute transfer by the attribute transfer unit 1080 and the geometric information reconstruction unit 1060 as input, and generate residual information for each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform). For example, the method described in the above-mentioned reference 2 can be used for the specific processing of RAHT.

[0380] The LoD calculation unit 1100 is configured to take geometric information generated by the geometric information reconstruction unit 1060 as input and generate LoD (Level of Detail).

[0381] LoD (Level of Data) is information used to define reference relationships (referring points and referenced points) for implementing predictive coding, which involves predicting the attribute information of one point from the attribute information of another point and then encoding or decoding the prediction residual.

[0382] In other words, LoD is information that defines a hierarchical structure in which each point included in geometric information is classified into multiple levels, and the attributes of points belonging to lower levels are encoded or decoded using the attribute information of points belonging to higher levels.

[0383] As for the specific method for determining the LoD, for example, the method described in the above-mentioned reference 2 may be used.

[0384] The lifting unit 1110 is configured to generate residual information through a lifting process using the LoD generated by the LoD calculation unit 1100 and the attribute information after attribute transfer in the attribute transfer unit 1080.

[0385] For specific lifting procedures, for example, the method described in the reference (Text of ISO / IEC 23090-9 DIS Geometry-based PCC, ISO / IEC JTC1 / SC29 / WG11 N19088) may be used.

[0386] The attribute information quantization unit 1120 is configured to quantize the residual information output from the RAHT unit 1090 or the lifting unit 1110. Here, when the quantization step size is 1, it is equivalent to not performing quantization.

[0387] The attribute information encoding unit 1130 is configured to encode the quantized residual information output from the attribute information quantization unit 1120 as syntax, and to generate a bitstream related to attribute information (attribute information bitstream).

[0388] The encoding process is, for example, context-adaptive binary arithmetic encoding. Here, for example, the syntax includes control data (flags and parameters) to control the decoding process of attribute information.

[0389] The point cloud encoding device 100 is configured to perform encoding processing on the positional information and attribute information of each point in the point cloud as input, and to output a geometric information bitstream and an attribute information bitstream.

[0390] Furthermore, the point cloud coding device 100 and point cloud decoding device 200 described above may be implemented as programs that cause a computer to execute each function (each process).

[0391] In the above embodiments, the present invention was described using the application to a point cloud coding device 100 and a point cloud decoding device 200 as an example. However, the present invention is not limited to such examples and can be similarly applied to a point cloud coding / decoding system equipped with the functions of the point cloud coding device 100 and the point cloud decoding device 200. [Industrial applicability]

[0392] Furthermore, according to this embodiment, for example, it is possible to achieve an overall improvement in service quality in video communication, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of Symbols]

[0393] 10…Point cloud processing system 100...Point cloud encoding device 1010... Coordinate transformation section 1020...Geometric information quantization section 1030...Tree Analysis Unit 1040…Approximate surface analysis section 1050...Geometric information encoding unit 1060...Geometric information reconstruction unit 1070...Color conversion unit 1080... Attribute Transfer Section 1090...RAHT Department 1100...LoD calculation unit 1110... Lifting Section 1120...Attribute information quantization section 1130...Attribute information encoding unit 200... Point cloud decoder 2010...Geometric Information Decoding Unit 2020... Tree Composition Section 2030…Approximate surface synthesis part 2040...Geometric information reconstruction unit 2050... Inverse coordinate transformation section 2060... Attribute Information Decoding Unit 2070...Inverse quantization section 2080…RAHT Department 2090...LoD calculation unit 2100... Reverse lifting section 2110... Reverse color conversion unit

Claims

1. A point cloud decoder, The system includes an approximate surface synthesis unit that decodes the node size to which Trisoup is applied to descendant nodes obtained by recursively dividing the node by Octree for each node of a predetermined size. The aforementioned approximate surface synthesis unit is a point cloud decoding device characterized by performing context-adaptive decoding using information on the presence or absence of vertices and vertex positions of spatially adjacent decoded segments when decoding the presence or absence of vertices and vertex positions of each segment constituting the Trisoup node.

2. The point cloud decoding apparatus according to claim 1, characterized in that when the approximate surface synthesis unit decodes the presence or absence of vertices and vertex positions of each segment constituting the Trisoup node, it performs context-adaptive decoding using information on the presence or absence of vertices and vertex positions of segments constituting nodes of the same size as the Trisoup node from among the spatially adjacent decoded segments.

3. The point cloud decoding device according to claim 1, further comprising a geometric information decoding unit that decodes the maximum Trisoup node size, which is the maximum node size to which the Trisoup is applied, and the minimum Trisoup node size, which is the minimum node size to which the Trisoup is applied.

4. The point cloud decoding device according to claim 3, characterized in that the geometric information decoding unit decodes the predetermined size as a value greater than or equal to the maximum TriSoup node size.

5. The point cloud decoding device according to claim 3, characterized in that the geometric information decoding unit sets the maximum Trisoup node size to the value of the predetermined size.

6. A point cloud decoding method, The process includes a step of decoding the node size to which Trisoup is applied to descendant nodes obtained by recursively dividing the node by Octree for each node of a predetermined size. A point cloud decoding method characterized in that, in the above step, when decoding the presence or absence of vertices and vertex positions of each segment constituting the Trisoup node, context-adaptive decoding is performed using information on the presence or absence of vertices and vertex positions of spatially adjacent decoded segments.

7. A program that makes a computer function as a point cloud decoder, The point cloud decoding device includes an approximate surface synthesis unit that decodes the node size to which Trisoup is applied to descendant nodes obtained by recursively dividing the node by Octree for each node of a predetermined size. The aforementioned approximate surface synthesis unit is a program characterized by performing context-adaptive decoding using information on the presence or absence of vertices and vertex positions of spatially adjacent decoded segments when decoding the presence or absence of vertices and vertex positions of each segment constituting the Trisoup node.

Images