Point group decoding device, point group decoding method and program

Abstract

To increase the efficiency of coding without compromising the subjective image quality of a decoding point group.SOLUTION: In a point group decoder 200 according to the present invention, an approximation surface synthesis unit 2030 executes decoding processing on the vertex in each node size between the largest size and the smallest size on the basis of the largest node size and the smallest node size decoded from control data, generates the position of the vertex in the smallest node size on the basis of the decoded vertex when the node size is not the smallest node size, and determines the vertex generated by a node with the largest node size of different nodes adjacent to each other to be the position of the vertex of a side when there are more than one vertices generated from nodes with different sizes adjacent to the side.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a point group decoding device, a point group decoding method, and a program. [Background technology]

[0002] Non-Patent Document 1 discloses a technique for reconstructing a decoded point group using a method called Trisoup for one predetermined type of node size. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] G-PCC Future Enhancement, ISO / IEC JTC1 / SC29 / WG11 N19328 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method of Non-Patent Document 1 has only one type of node size (fixed node size), and there is a problem that there is room for improvement in terms of coding efficiency.

[0005] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a point cloud decoding device, a point cloud decoding method, and a program that can improve encoding efficiency without compromising the subjective image quality of the decoded point cloud. [Means for solving the problem]

[0006] A first feature of the present invention is a point cloud decoding device comprising an approximate surface synthesis unit, which performs a vertex decoding process at each node size between a maximum and a minimum based on a maximum node size and a minimum node size decoded from control data, and if the node size is not the minimum node size, generates a vertex position at the minimum node size based on the decoded vertex, and if there are multiple vertices on a certain side generated from nodes of different node sizes adjacent to the certain side, the vertex generated by the node with the largest node size among the different adjacent nodes is configured to be the vertex position of the certain side.

[0007] A second feature of the present invention is a point cloud decoding method comprising the steps of: executing a vertex decoding process at each node size between a maximum and a minimum based on a maximum node size and a minimum node size decoded from control data; if the node size is not the minimum node size, generating a vertex position at the minimum node size based on the decoded vertex; and, if there are multiple vertices on a certain side generated from nodes of different node sizes adjacent to the certain side, setting the vertex generated by the node with the largest node size among the different adjacent nodes as the vertex position of the certain side.

[0008] A third feature of the present invention is a program for causing a computer to function as a point cloud decoding device, the point cloud decoding device comprising an approximate surface synthesis unit, which performs a vertex decoding process at each node size between a maximum and a minimum based on a maximum node size and a minimum node size decoded from control data, and if the node size is not the minimum node size, generates a vertex position at the minimum node size based on the decoded vertex, and if there are multiple vertices on a certain edge generated from nodes of different node sizes adjacent to the certain edge, the vertex generated by the node with the largest node size among the different adjacent nodes is configured to be the vertex position of the certain edge. Effect of the Invention

[0009] According to the present invention, it is possible to provide a point group decoding device, a point group decoding method, and a program that can improve encoding efficiency without compromising the subjective image quality of the decoded point group. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a point cloud processing system 10 according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of functional blocks of a point group decoding device 200 according to an embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of the configuration of encoded data (bit stream) received by the geometric information decoding unit 2010 of the point cloud decoding device 200 according to an embodiment. [Figure 4] FIG. 4 is a diagram showing an example of the syntax configuration of GPS2011. [Diagram 5] FIG. 5 is a diagram showing an example of the syntax configuration of GSH2012. [Figure 6] FIG. 6 is a diagram showing an example of the syntax configuration of GSH2012. [Figure 7] FIG. 7 is a diagram showing an example of the syntax configuration of GSH2012. [Figure 8] FIG. 8 is a flowchart illustrating an example of a process in the tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of processing in the tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of the process of the approximate surface synthesis unit 2030 of the point cloud decoding device 200 according to an embodiment. [Figure 11A] FIG. 11A is a diagram for explaining an example of the process in step S1006 in FIG. [Figure 11B]FIG. 11B is a diagram for explaining an example of the process in step S1006 in FIG. [Figure 12A] FIG. 12A is a diagram for explaining an example of the process in step S1006 in FIG. [Figure 12B] FIG. 12B is a diagram for explaining an example of the process in step S1006 in FIG. [Figure 13] FIG. 13 is a flowchart showing an example of the vertex decoding process used in step S1002 in FIG. [Figure 14] FIG. 14 is a flowchart showing an example of the process in step S1304 in FIG. [Figure 15] FIG. 15 is a diagram showing an example of functional blocks of the point group encoding device 100 according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0012] (First embodiment) A point cloud processing system 10 according to a first embodiment of the present invention will be described below with reference to Fig. 1 to Fig. 15. Fig. 1 is a diagram showing a point cloud processing system 10 according to the embodiment.

[0013] As shown in FIG. 1, a point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0014] The point cloud encoding device 100 is configured to generate encoded data (bit stream) by encoding an input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bit stream.

[0015] The input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud, such as color information and reflectance of each point.

[0016] Here, such a bit stream may be transmitted from the point group encoding device 100 to the point group decoding device 200 via a transmission path. Also, the bit stream may be stored in a storage medium and then provided from the point group encoding device 100 to the point group decoding device 200.

[0017] (Point Cloud Decoding Device 200) Hereinafter, the point group decoding device 200 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the point group decoding device 200 according to this embodiment.

[0018] As shown in FIG. 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, an LoD calculation unit 2090, an inverse lifting unit 2100, and an inverse color transformation unit 2110.

[0019] The geometric information decoding unit 2010 is configured to receive as input a bit stream relating to geometric information (geometric information bit stream) out of the bit streams output from the point group encoding device 100, and to decode the syntax.

[0020] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0021] The tree synthesis unit 2020 is configured to receive as input the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating which node in the tree described below the point group exists in, and generate tree information indicating in which area in the space to be decoded the point exists.

[0022] The tree synthesis unit 2020 may be configured to perform the decoding process of the occupancy code within itself.

[0023] This process divides the space to be decoded into rectangles, determines whether a point exists in each rectangle by referring to the occupancy code, divides the rectangle in which the point exists into multiple rectangles, and then generates tree information by recursively repeating the process of referring to the occupancy code.

[0024] Here, when decoding the occupancy code, inter prediction, which will be described later, may be used.

[0025] In this embodiment, a method called "Octree" can be used, which always treats the above-mentioned rectangular parallelepiped as a cube and performs octree division recursively, and a method called "QtBt" can be used, which performs quadtree division and binary tree division in addition to octree division. Whether or not to use "QtBt" is transmitted from the point cloud encoding device 100 as control data.

[0026] In this embodiment, the method of decoding geometric information by dividing the above-mentioned cube by "Octree" until it has a size of 1x1x1 is specifically referred to as "Octree only."

[0027] In addition, even if "Octree" is used in combination with "QtBt", the method of decoding geometric information by dividing the above-mentioned rectangular prism into 1x1x1 size using only "Octree" and "QtBt" may also be called "Octree only".

[0028] Alternatively, when the use of predictive coding is specified by the control data, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree structure determined in the point cloud encoding device 100.

[0029] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree synthesis unit 2020, and to decode the point cloud based on the approximate surface information.

[0030] Approximate surface information is used when, for example, decoding three-dimensional point cloud data of an object, in cases where the point cloud is densely distributed on the object surface, to approximate the area in which the point cloud exists using a small plane, rather than decoding each individual point cloud.

[0031] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information and decode the point cloud using a method called "Trisoup", for example. A specific processing example of "Trisoup" will be described later. In addition, when decoding a sparse point cloud acquired by Lidar or the like, this processing can be omitted.

[0032] The geometric information reconstruction unit 2040 is configured to reconstruct geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point cloud data to be decoded, based on the tree information generated by the tree synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0033] The inverse coordinate transformation unit 2050 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform the information from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

[0034] The attribute information decoding unit 2060 is configured to receive as input a bit stream relating to attribute information (attribute information bit stream) out of the bit streams output from the point group encoding device 100, and to decode the syntax.

[0035] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0036] Moreover, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

[0037] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and a quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, to generate inverse quantized residual information.

[0038] The dequantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090 according to the features of the point group to be decoded. Control data decoded by the attribute information decoding unit 2060 specifies which unit the dequantized residual information is output to.

[0039] The RAHT unit 2080 is configured to receive the inverse quantized residual information generated by the inverse quantization unit 2070 and the geometric information generated by the geometric information reconstruction unit 2040 as input, and to decode the attribute information of each point using a type of Haar transform (inverse Haar transform in the decoding process) called RAHT (Region Adaptive Hierarchical Transform). As a specific process of RAHT, for example, the method described in Non-Patent Document 1 can be used.

[0040] The LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 as an input and generate a Level of Detail (LoD).

[0041] LoD is information for defining a reference relationship (a referencing point and a referenced point) to realize predictive coding, such as predicting attribute information of a certain point from attribute information of another point and encoding or decoding the prediction residual.

[0042] In other words, LoD is information that defines a hierarchical structure in which each point contained in geometric information is classified into multiple levels, and attributes of points belonging to lower levels are encoded or decoded using attribute information of points belonging to higher levels.

[0043] As a specific method for determining the LoD, for example, the method described in Non-Patent Document 1 mentioned above may be used.

[0044] The inverse lifting unit 2100 is configured to decode attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse quantized residual information generated by the inverse quantization unit 2070. As a specific process of inverse lifting, for example, the method described in the above-mentioned Non-Patent Document 1 can be used.

[0045] The inverse color conversion unit 2110 is configured to perform inverse color conversion processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information to be decoded is color information and color conversion has been performed on the point cloud encoding device 100 side. Whether or not to perform such inverse color conversion processing is determined by the control data decoded by the attribute information decoding unit 2060.

[0046] The point group decoding device 200 is configured to decode and output the attribute information of each point in the point group through the above processing.

[0047] (Geometric Information Decoding Part 2010) The control data decoded by the geometric information decoding unit 2010 will be described below with reference to FIGS.

[0048] FIG. 3 shows an example of the structure of the coded data (bit stream) received by the geometric information decoding unit 2010. In FIG.

[0049] First, the bit stream may include a GPS 2011. The GPS 2011 is also called a geometry parameter set, and is a set of control data related to decoding of geometric information. A specific example will be described later. Each GPS 2011 includes at least GPS ID information for identifying each GPS 2011 when there are multiple GPS 2011s.

[0050] Secondly, the bit stream may include GSH2012A / 2012B. GSH2012A / 2012B is also called a geometry slice header or geometry data unit header, and is a set of control data corresponding to a slice described later. In the following description, the term "slice" is used, but slice can also be read as data unit. A specific example will be described later. GSH2012A / 2012B includes at least GPS id information for specifying GPS2011 corresponding to each GSH2012A / 2012B.

[0051] Thirdly, the bit stream may include slice data 2013A / 2013B following the GSH 2012A / 2012B. The slice data 2013A / 2013B includes data that encodes geometric information. An example of the slice data 2013A / 2013B is an occupancy code, which will be described later.

[0052] As described above, the bit stream is configured such that each slice data 2013A / 2013B corresponds to one GSH 2012A / 2012B and one GPS 2011.

[0053] As described above, since the GPS ID information is used to specify which GPS 2011 to refer to in the GSH 2012A / 2012B, a common GPS 2011 can be used for a plurality of slice data 2013A / 2013B.

[0054] In other words, it is not necessary to transmit the GPS 2011 for each slice. For example, as shown in Fig. 3, the bit stream may be configured such that the GPS 2011 is not coded immediately before the GSH 2012B and the slice data 2013B.

[0055] 3 is merely an example. As long as the slice data 2013A / 2013B corresponds to the GSH 2012A / 2012B and the GPS 2011, elements other than those described above may be added as components of the bit stream.

[0056] For example, as shown in Fig. 3, the bitstream may include a sequence parameter set (SPS) 2001. Similarly, when transmitted, the bitstream may be shaped into a configuration different from that shown in Fig. 3. Furthermore, the bitstream may be combined with a bitstream decoded by an attribute information decoding unit 2060 (described later) and transmitted as a single bitstream.

[0057] FIG. 4 is an example of the syntax configuration of GPS2011.

[0058] Note that the syntax names described below are merely examples. If the syntax functions described below are similar, the syntax names may be different.

[0059] The GPS 2011 may include GPS ID information (gps_geom_parameter_set_id) for identifying each GPS 2011.

[0060] The Descriptor column in Fig. 4 indicates how each syntax is coded. ue(v) indicates an unsigned zeroth-order exponential Golomb code, and u(1) indicates a 1-bit flag.

[0061] The GPS 2011 may include a flag (trisoup_enabled_flag) that controls whether or not Trisoup is used in the approximate surface synthesis unit 2030 .

[0062] For example, it may be defined that Trisoup is not used when the value of trisoup_enabled_flag is "0", and that Trisoup is used when the value of trisoup_enabled_flag is "1".

[0063] The geometric information decoding unit 2020 may be configured to additionally decode the following syntax when Trisoup is used, that is, when the value of trisoup_enabled_flag is “1”.

[0064] Note that trisoup_enabled_flag may be included in SPS2001 instead of GPS2011.

[0065] GPS2011 may include a flag (trisoup_multilevel_enabled_flag, first flag) that controls whether or not Trisoup is permitted on multiple levels.

[0066] For example, when the value of trisoup_multilevel_enabled_flag is "0", multi-level trisoup is not allowed, i.e., a single-level trisoup is performed, and when the value of trisoup_multilevel_enabled_flag is "1", multi-level trisoup is allowed.

[0067] If the syntax is not included in GPS2011, the value of the syntax may be regarded as the value when performing Trisoup at a single level, i.e., "0".

[0068] Note that trisoup_multilevel_enabled_flag may be defined to be included in SPS2001 instead of GPS2011. In this case, if trisoup_multilevel_enabled_flag is not included in SPS2001, the value of the syntax may be regarded as the value for performing trisoup at a single level, i.e., "0".

[0069] Figure 5 shows an example of the syntax configuration of GSH 2012. As mentioned above, GSH is also called GDUH (Geometry Data Unit Header).

[0070] The geometric information decoding unit 2020 may be configured to additionally decode the following syntax when multi-level Trisoup is permitted, that is, when the value of trisoup_multilevel_enabled_flag is "1".

[0071] GSH2012 may include a flag (adaptive_octree_enabled_flag) that enables a mode that adaptively selects between octree-only and trisoup within a slice when allowing trisoup at multiple levels.

[0072] For example, it may be defined that when the value of the flag is "1", a mode that adaptively selects Octree and Trisoup (hereinafter referred to as "adaptive Octree mode") is enabled, and when the value of the flag is "0", the adaptive Octree mode is disabled.

[0073] Also, if GSH2012 does not include this flag, the value of the flag may be implicitly considered to be "0", i.e., adaptive octree mode is disabled.

[0074] When GSH2012 allows multi-level trisoups, it may include syntax (log2_trisoup_max_node_size_minus2) to specify the maximum trisoup node size.

[0075] The syntax may be expressed as a value obtained by converting the maximum value of the actual Trisoup node size into a logarithm with the base 2. Furthermore, the syntax may be expressed as a value obtained by subtracting 2 from the logarithm with the maximum value of the actual Trisoup node size.

[0076] GSH2012 may include syntax (log2_trisoup_min_node_size_minus2) to specify a minimum Trisoup node size when allowing multi-level Trisoups.

[0077] The syntax may be expressed as a value obtained by converting the minimum value of the actual Trisoup node size to a logarithm with the base 2. Furthermore, the syntax may be expressed as a value obtained by subtracting 2 from the logarithm obtained by converting the minimum value of the actual Trisoup node size to 4 (=2^2).

[0078] In addition, the value of this syntax may be restricted to be greater than or equal to 0 and less than or equal to log2_trisoup_max_node_size_minus2.

[0079] In this case, trisoup_depth may be defined as trisoup_depth=log2_trisoup_max_node_size_minus2-log2_trisoup_min_node_size_minus2+1 as shown in FIG.

[0080] Also, instead of directly decoding the minimum Trisoup node size and the maximum Trisoup node size, a Depth value corresponding to the maximum Trisoup node size and the minimum Trisoup node size in Octree processing described later may be decoded.

[0081] For example, if the maximum depth (the depth at which all nodes are 1x1x1 in size) is 10, the minimum Trisoup node size is 4 (=2 2 ), and the maximum Trisoup node size is 16 (=2 4 ), then 8 may be decoded as the Depth value corresponding to the minimum Trisoup node size, and 6 may be decoded as the Depth value corresponding to the maximum Trisoup node size.

[0082] The geometric information decoding unit 2020 may be configured to additionally decode the following syntax when multi-level Trisoup is not permitted, that is, when the value of trisoup_multilevel_enabled_flag is "0".

[0083] GSH2012 may include syntax (log2_trisoup_node_size_minus2) to specify the Trisoup node size when multi-level Trisoup is not allowed and when Trisoup is used.

[0084] The syntax may be expressed as a value obtained by converting the actual Trisoup node size to a logarithm with the base 2. Furthermore, the syntax may be expressed as a value obtained by subtracting 2 from the logarithm with the actual Trisoup node size.

[0085] In this case, trisoup_depth may be defined as trisoup_depth=1 as shown in FIG.

[0086] When using Trisoup, GSH2012 may include a syntax (trisoup_sampling_value_minus1) that controls the sampling interval of the decoding point. The specific definition of the syntax may be the same as that described in, for example, the above-mentioned Non-Patent Document 1.

[0087] When GSH2012 uses Trisoup and allows Trisoup at multiple levels, it may include a flag (unique_segments_exist_flag[i]) for each hierarchy i (i=0,...,trisoup_depth-1) indicating whether unique segments exist in the target hierarchy.

[0088] For example, if the value of unique_segments_exist_flag[i] is "1", it means that at least one unique segment exists in hierarchical layer i. If the value of unique_segments_exist_flag[i] is "0", it means that no unique segments exist in hierarchical layer i.

[0089] If a unique segment exists in the target hierarchical layer for each hierarchical layer i (i=0,...,trisoup_depth-1), i.e., if the value of unique_segments_exist_flag[i] is "1", GSH2012 may additionally include syntax (num_unique_segments_bits_minus1[i]) indicating the number of bits of the syntax indicating the number of unique segments in the target hierarchical layer and syntax (num_unique_segments_minus1[i]) indicating the number of unique segments in the target hierarchical layer.

[0090] Here, for both num_unique_segments_bits_minus1[i] and num_unique_segments_minus1[i], the original value minus "1" may be coded as the syntax value.

[0091] Figure 6 shows an example of the syntax configuration of GSH 2012. In the following, only the differences from Figure 5 will be explained.

[0092] As shown in FIG. 6, when the adaptive octree mode is enabled, that is, when the value of adaptive_octree_enabled_flag is "1", decoding of the syntax (log2_trisoup_min_node_size_minus2) that specifies the minimum value of the Trisoup node size may be omitted. In this case, the minimum value of the Trisoup node size may be regarded as a predetermined value. For example, the predetermined value may be "0" (i.e., 2 0 = 1).

[0093] As described above, the geometric information decoding unit 2010 may be configured to decode a flag that controls whether or not both Octree and Trisoup can be used within the same slice or data unit, and if the value of the flag indicates “usable,” to implicitly regard the value of the minimum Trisoup node size as 1×1×1.

[0094] By adopting the above-mentioned configuration, it is possible to eliminate the transmission of unnecessary syntax and reduce the amount of code.

[0095] Figure 7 shows an example of the syntax configuration of GSH 2012. In the following, only the differences from Figure 5 will be explained.

[0096] As shown in FIG. 7, for each of the syntax that specifies the maximum value of the Trisoup node size and the syntax that specifies the minimum value of the Trisoup node size, the values ​​converted to logarithms with the base 2 may be transmitted. In this case, the minimum value of each syntax may be set to "0" (i.e., 2 0 = 1).

[0097] As described above, the geometric information decoding unit 2010 may be configured to decode the minimum Trisoup node size, and the minimum value that the minimum Trisoup node size can take may be a value equivalent to 1×1×1.

[0098] By adopting such a configuration, the processing can be standardized when only Trisoup is used.

[0099] (Tree Synthesis Department 2020) The processing of the tree merging unit 2020 will be described below with reference to Fig. 8 and Fig. 9. Fig. 6 is a flowchart showing an example of the processing in the tree merging unit 2020. Note that, below, an example of merging trees using "Octree" will be described.

[0100] In step S801, the tree synthesis unit 2020 checks whether the processing of all depths has been completed. Note that the depth number may be included as control data in a bit stream transmitted from the point group encoding device 100 to the point group decoding device 200.

[0101] The tree synthesis unit 2020 calculates the node size of the target depth. In the case of an "Octree", the node size of the first depth may be defined as "2 to the power of the depth number". In other words, if the depth number is N, the node size of the first depth may be defined as 2 to the power of N.

[0102] In addition, the node size at the second and subsequent depths may be defined by decreasing the number N by 1. That is, the node size at the second depth may be defined as "2 to the power of (N-1)", the node size at the third depth may be defined as "2 to the power of (N-2)", and so on.

[0103] Alternatively, since the node size is always defined as a power of 2, the value of the exponent part (N, N-1, N-2, etc.) may simply be considered as the node size. In the following explanation, the node size refers to the value of the exponent part of the length of one side of the node.

[0104] For simplicity, the following description will be given taking as an example a case where the node shape is a cube, that is, where all sides of the node have the same length.

[0105] When using QtBt, that is, when the node shape is a rectangular parallelepiped and the length of each side of the node differs for each axis direction (x, y, z), the length of the shortest side among the three directions may be considered as the node size. Similarly, the length of the longest side among the three directions may be considered as the node size.

[0106] Here, when the flag (trisoup_enabled_flag) that controls whether to use Trisoup indicates that Trisoup is used, that is, when the value of trisoup_enabled_flag is "1", the tree synthesis unit 2020 may change the number of Depths to be processed based on the value of the syntax (log2_trisoup_min_node_size_minus2) that specifies the minimum value of the Trisoup node size or the syntax (log2_trisoup_node_size_minus2) that specifies the Trisoup node size. In such a case, for example, it may be defined as follows.

[0107] Processing Depth Number = Total Depth Number - (Minimum) Trisoup Node Size Here, the minimum Trisoup node size can be defined as (log2_trisoup_min_node_size_minus2+2), for example. Similarly, the Trisoup node size can be defined as (log2_trisoup_node_size_minus2+2).

[0108] In this case, if the processing for all the processing depths has been completed, the tree merging unit 2020 proceeds to step S809, and if not, the tree merging unit 2020 proceeds to step S802.

[0109] In other words, if (number of processing depths-n)=0, the tree merging unit 2020 proceeds to step S809, and if (number of processing depths-n)>0, the tree merging unit 2020 proceeds to step S802.

[0110] Furthermore, the tree synthesis unit 2020 may determine that Trisoup is applied to all nodes having the node size (N--the number of processing depths) when proceeding to step S809.

[0111] Here, when multi-level Trisoup is permitted, i.e., when the value of trisoup_multilevel_enabled_flag is "1", and when adaptive Octree mode is enabled, i.e., when the value of adaptive_octree_enabled_flag is "1", step S801 may be configured as follows.

[0112] Specifically, if the processing of all depths has been completed, the tree composing unit 2020 proceeds to step S809, and if the processing of all depths has not been completed, the tree composing unit 2020 proceeds to step S802.

[0113] In other words, when the target depth is the nth depth, if (Nn)=0, the tree composing unit 2020 proceeds to step S809, and if (Nn)>0, the tree composing unit 2020 proceeds to step S802.

[0114] In step S802, the tree synthesis unit 2020 determines whether or not it is necessary to decode a Trisoup_applied_flag, which will be described later, at the target depth.

[0115] For example, if "multiple levels of Trisoup are allowed (the value of trisoup_multilevel_enabled_flag is "1")" and "the node size (Nn) of the target depth is less than or equal to the maximum Trisoup node size," the tree synthesis unit 2020 may determine that "decoding of Trisoup_applied_flag is necessary."

[0116] For example, if "multiple levels of Trisoup are allowed (the value of trisoup_multilevel_enabled_flag is "1")" and "the node size (Nn) of the target depth is less than or equal to the maximum Trisoup node size" and "the node size (Nn) of the target depth is greater than or equal to the minimum Trisoup node size", the tree synthesis unit 2020 may determine that "decoding of Trisoup_applied_flag is necessary."

[0117] Furthermore, the tree synthesis unit 2020 may determine that "decoding of Trisoup_applied_flag is not necessary" if the above-mentioned condition is not satisfied.

[0118] Here, the maximum Trisoup node size can be defined as, for example, (log2_trisoup_max_node_size_minus3+2), and the minimum Trisoup node size can be defined as, for example, (log2_trisoup_min_node_size_minus2+2).

[0119] When the above-mentioned determination is completed, the tree synthesis unit 2020 proceeds to step S803.

[0120] In step S803, the tree synthesis unit 2020 determines whether or not the processing of all nodes included in the target depth has been completed.

[0121] If it is determined that the processing of all nodes of the target depth has been completed, the tree merging unit 2020 proceeds to step S801 and performs processing of the next depth.

[0122] On the other hand, if processing of all nodes at the target depth has not been completed, the tree merging unit 2020 proceeds to step S804.

[0123] In step S804, the tree synthesis unit 2020 checks whether or not the Trisoup_applied_flag determined in step S802 needs to be decoded.

[0124] If it is determined that the Trisoup_applied_flag needs to be decoded, the tree merging unit 2020 proceeds to step S805, and if it is determined that the Trisoup_applied_flag needs not to be decoded, the tree merging unit 2020 proceeds to step S808.

[0125] In step S805, the tree synthesis unit 2020 decodes Trisoup_applied_flag.

[0126] Trisoup_applied_flag is a 1-bit flag (second flag) indicating whether Trisoup is applied to the target node. For example, it may be defined that Trisoup is applied to the target node when the value of this flag is "1", and that Trisoup is not applied to the target node when the value of this flag is "0".

[0127] After decoding Trisoup_applied_flag, the tree synthesis unit 2020 proceeds to step S806.

[0128] In step S806, the tree synthesis unit 2020 checks the value of Trisoup_applied_flag decoded in step S805.

[0129] If Trisoup is to be applied to the target node, that is, if the value of Trisoup_applied_flag is "1", the tree synthesis unit 2020 proceeds to step S807.

[0130] If Trisoup is not applied to the target node, that is, if the value of Trisoup_applied_flag is "0", the tree synthesis unit 2020 proceeds to step S808.

[0131] In step S807, the tree synthesis unit 2020 stores the target node as a node to which Trisoup is applied, that is, as a Trisoup node. No further node division by "Octree" is applied to the target node. After that, the tree synthesis unit 2020 proceeds to step S803 and moves to processing of the next node.

[0132] In step S808, the tree synthesis unit 2020 decodes information called an occpancy code.

[0133] In the case of an "Octree", the occpancy code is information that indicates whether the point to be decoded is contained in each of the child nodes when the target node is divided in half in each of the x, y, and z directions and divided into eight nodes (called child nodes).

[0134] For example, the occpancy code may assign one bit of information to each child node, and if the one bit of information is "1", it is defined that the point to be decoded is contained within the child node, and if the one bit of information is "0", it is defined that the point to be decoded is not contained within the child node.

[0135] When decoding such an occpancy code, the tree synthesis unit 2020 may estimate in advance the probability that the point to be decoded exists in each child node, and entropy decode the bits corresponding to each child node based on that probability.

[0136] Similarly, in the point cloud encoding device 100, entropy encoding may be performed.

[0137] As described above, the point cloud decoding device 200 may be configured to use both Octree only and Trisoup within the same slice or data unit, and to decode geometric information of points using either the Octree only or Trisoup method for each node.

[0138] The above configuration makes it possible to switch between using only Octree for areas with locally detailed geometric structures and Trisoup for areas that can be approximated by a plane, enabling efficient coding.

[0139] As described above, the tree synthesis unit 2020 may be configured to perform Octree division to recursively divide the space, and to decode the geometric information for all nodes that are smaller than a predetermined minimum Trisoup node size using only the Octree.

[0140] As described above, when the adaptive Octree mode is enabled, the tree synthesis unit 2020 may be configured to decode, for each node of a predetermined minimum Trisoup node size, a flag (Trisoup_applied_flag) indicating whether or not Trisoup is applied to the node.

[0141] As described above, the geometric information decoding unit 2010 may be configured to decode a flag that controls whether or not both Octree and Trisoup can be used within the same slice or data unit, and when the value of the flag indicates “usable,” the tree synthesis unit 2020 may be configured to decode the geometric information for all nodes that are smaller than a predetermined minimum Trisoup node size using only Octree.

[0142] With the above configuration, when the Trisoup node size is at its minimum, it can be used in conjunction with Octree alone without making any changes from Non-Patent Document 1, which has the advantage of reducing the amount of changes to the processing of Trisoup itself and making design easier.

[0143] Fig. 9 is a flowchart showing an example of processing in the tree merging unit 2020. The same reference numerals are used to denote parts that perform the same processing as in Fig. 8, and a description thereof will be omitted.

[0144] In step S901, if the processing for all the processing depths has been completed, the tree merging unit 2020 proceeds to step S809, and if not, the tree merging unit 2020 proceeds to step S802.

[0145] Here, when multi-level Trisoup is permitted (the value of trisoup_multilevel_enabled_flag is "1"), the tree synthesis unit 2020 may change the number of Depths to be processed based on the value of the syntax (log2_trisoup_node_size shown in FIG. 7) that specifies the minimum value of the Trisoup node size. At this time, the tree synthesis unit 2020 may set the minimum value of the Trisoup node size to 0.

[0146] 6, when the adaptive octree mode is enabled, the tree synthesis unit 2020 may implicitly set the minimum value of the Trisoup node size to 0. In this case, for example, it may be defined as follows.

[0147] Processing Depth Number = Total Depth Number - Minimum Trisoup Node Size In this case, if the processing for all the processing depths has been completed, the tree merging unit 2020 proceeds to step S809, and if not, the tree merging unit 2020 proceeds to step S902.

[0148] In other words, if (number of processing depths-n)=0, the tree merging unit 2020 proceeds to step S809, and if (number of processing depths-n)>0, the tree merging unit 2020 proceeds to step S902.

[0149] In step S902, the tree synthesis unit 2020 determines whether or not it is necessary to decode a Trisoup_applied_flag, which will be described later, at the target depth.

[0150] For example, if "multiple levels of Trisoup are allowed (the value of trisoup_multilevel_enabled_flag is "1")" and "the node size (Nn) of the target depth is less than or equal to the maximum Trisoup node size," the tree synthesis unit 2020 may determine that "decoding of Trisoup_applied_flag is necessary."

[0151] In addition, the tree synthesis unit 2020 determines whether the node size (N-processing depth number) is 1 or more (2 1 , it may be determined that Trisoup applies to all nodes with that node size.

[0152] In addition, the tree synthesis unit 2020 determines whether the node size (N-processing depth number) is 0 (2 0 = 1), that is, in the case of a node of size 1x1x1, the approximate surface synthesis unit 2030 described later does not perform Trisoup processing, and may regard the 1x1x1 node itself as a decoded point, as in the case of executing only Octree.

[0153] As described above, the tree synthesis unit 2020 may be configured to set a predetermined minimum Trisoup node size to 1, and determine that for nodes with a node size of 1×1×1, the geometric information is decoded only by Octree, and for nodes with a node size larger than 1×1×1, the geometric information is decoded by Trisoup.

[0154] The above-mentioned configuration has the advantage that the processing can be shared with the case where only Trisoup is used.

[0155] (Approximate surface synthesis section 2030) The approximate surface synthesis unit 2030 is configured to perform a decoding process for each node determined to be a Trisoup node by the tree synthesis unit 2020, as will be described with reference to Figs.

[0156] An example of the processing of the approximate surface synthesis unit 2030 will be described below with reference to FIGS.

[0157] FIG. 10 is a flowchart showing an example of the process of the approximate surface synthesis unit 2030.

[0158] As shown in FIG. 10, in step S1001, the approximate surface synthesis unit 2030 determines whether or not the processing at all trisoup_depths has been completed.

[0159] If the processing has been completed for all trisoup_depth, the process proceeds to step S1009, where the processing ends. If the processing has not been completed for all trisoup_depth, the process proceeds to step S1002.

[0160] In step S1002, the approximate surface synthesis unit 2030 obtains and integrates the vertex positions for each node.

[0161] First, the approximate surface synthesis unit 2030 decodes the vertex positions for each node in the Trisoup node size corresponding to the trisoup_depth (hereinafter, referred to as the Trisoup node size). Specific processing will be described with reference to FIG.

[0162] Secondly, if the Trisoup node size is smaller than the maximum Trisoup node size, the approximate surface synthesis unit 2030 performs integration processing with vertices generated in a Trisoup node size larger than the Trisoup node size in step S1008 described later.

[0163] Specifically, when both a vertex decoded by the method described in Fig. 13 (a vertex in the Trisoup node size) and a vertex generated by the method described in step S1008 (a vertex generated in a Trisoup node size larger than the Trisoup node size) exist on an edge of a node in the Trisoup node size, the approximate surface synthesis unit 2030 performs processing to hold only the vertex generated in the largest node size and delete the other vertices. In this way, it is possible to reduce the number of vertices per node to a maximum of one.

[0164] After the vertex positions have been obtained and integrated, the approximate surface synthesis unit 2030 proceeds to step S1003.

[0165] In step S1003, the approximate surface synthesis unit 2030 determines a projection plane for each node in the Trisoup node size.

[0166] For example, when there are three axes, x-axis, y-axis, and z-axis, a plane with one of the axes degenerated is called a projection plane.

[0167] In step S1003, the approximate surface synthesis unit 2030 determines which of the above-mentioned axes to degenerate, that is, which of the xy plane, xz plane, and yz plane to use as the projection plane. A specific method for determining the projection plane is omitted because a known method can be applied.

[0168] After determining the projection surface, the approximate surface synthesis unit 2030 proceeds to step S1004.

[0169] In step S1004, the approximate surface synthesis unit 2030 sorts the vertices projected onto the projection surface, for example, in a counterclockwise order, and assigns indices according to the order. If the sorting process is performed during the process of step S1003, the process of step S1004 can be omitted by saving the result of the sorting process.

[0170] After assigning the index, the approximate surface synthesis unit 2030 proceeds to step S1005.

[0171] In step S1005, the approximate surface synthesis unit 2030 generates triangles based on the above-mentioned index and the number of vertices existing in the target node.

[0172] For example, the approximate surface synthesis unit 2030 can generate triangles by creating a table that defines which index a triangle is generated from for each number of vertices in advance and referring to the table. As a specific example of the table, for example, the table described in the above-mentioned Non-Patent Document 1 can be used.

[0173] Alternatively, the approximate surface synthesis unit 2030 may first generate a centroid by averaging the coordinates of each vertex, and then generate a triangle using the two adjacent points in the above-mentioned index order and the centroid, for a total of three points.

[0174] After generating the triangles, the approximate surface synthesis unit 2030 proceeds to step S1006.

[0175] In step S1006, the approximate surface synthesis unit 2030 generates points based on the triangles generated in step S1005. As a specific method, for example, the method described in Non-Patent Document 1 mentioned above can be used.

[0176] After the generation of points for all nodes is completed, the approximate surface synthesis unit 2030 proceeds to step S1007.

[0177] In step S1007, the approximate surface synthesis unit 2030 determines whether the Trisoup node size is equal to the minimum node size.

[0178] If the Trisoup node size is equal to the minimum node size, the approximate surface synthesis unit 2030 proceeds to step S1001 to perform the next process of trisoup_depth, and if not, proceeds to step S1008.

[0179] In step S1008, the approximate surface synthesis unit 2030 generates vertices in the minimum Trisoup node size based on the points in the Trisoup node size generated in step S1006 or the triangles generated in step S1005.

[0180] FIG. 11 shows an example of the case where vertices in the minimum Trisoup node size are generated from the point group in the Trisoup node size generated in step S1006.

[0181] FIG. 11A shows an example of the decoded point group generated in step S1006.

[0182] First, the approximate surface synthesis unit 2030 divides the node into portions of the minimum Trisoup node size, as shown by the dotted lines in FIG. 11A.

[0183] In FIG. 11, the Trisoup node size is N(=2 N ) and the minimum Trisoup node size is N-1(=2 (N-1) ) is shown.

[0184] Secondly, the approximate surface synthesis unit 2030 checks whether each point in the decoded point group exists on each edge (solid line and dotted line in FIG. 11A) when divided for each minimum Trisoup node size.

[0185] For example, if each edge is 2 (N-1) If we consider that the edges exist at intervals, the coordinates of the edges in the z-axis direction are (A×2 (N-1) , B×2 (N-1) ,z), where A and B are constants.

[0186] Therefore, for example, if the coordinates of a point in the decoded point group are x=A×2 (N-1) and y = B × 2 (N-1) If there is a point such that: , then that point can be regarded as a point existing on an edge. The above process can be similarly applied to edges in the x-axis direction and edges in the y-axis direction.

[0187] The approximate surface synthesis unit 2030 performs such processing for all edges when divided into each minimum Trisoup node size, thereby making it possible to extract points existing on edges from the decoded point group.

[0188] The above-mentioned process is merely an example, and other processes may be used as long as they are capable of extracting points on edges in the minimum Trisoup node size.

[0189] For example, if each edge is 2 (N-1) When it is assumed that the points exist at intervals, the approximate surface synthesis unit 2030 extracts points existing on the edge in the z-axis direction by multiplying the x-coordinate and the y-coordinate of each point in the decoded point group by 2x. (N-1) It is also possible to extract points that have zero remainders in both the x and y coordinates as points that exist on an edge.

[0190] Fig. 11B shows an example of the extraction result of points on the edge. As shown in Fig. 11B, the approximate surface synthesis unit 2030 may extract only points on the edge that exist on the surface of the Trisoup node size, and may exclude points on the edge that exist inside the Trisoup node size from the extraction target.

[0191] The approximate surface synthesis unit 2030 stores the points on the edges extracted in this manner as vertices in the minimum Trisoup node size.

[0192] As described above, the approximate surface synthesis unit 2030 may be configured to perform a vertex decoding process at each node size between the maximum and minimum based on the maximum node size and minimum node size decoded from the control data, and if the node size is not the minimum node size, to generate a vertex position at the minimum node size based on the decoded vertex, and if there are multiple vertices on a certain edge generated from nodes of different node sizes adjacent to the edge, to set the vertex generated by the node with the largest node size among the adjacent nodes as the vertex position of the edge.

[0193] This configuration eliminates gaps in the mesh even at boundaries of different node sizes, which is expected to improve the subjective quality of the decoded point cloud.

[0194] Furthermore, as described above, when generating vertex positions for a minimum node size, vertices may be generated only on the surface of the node.

[0195] As described above, by limiting the vertex positions to be generated only to node surfaces that may be adjacent to different node sizes, it is possible to avoid generating unnecessary vertices and prevent an increase in the amount of processing.

[0196] As described above, the approximate surface synthesis unit 2030 may be configured to, when the node size is not the minimum node size, set, among the points in the decoded point group of the target node, the points on the edges when the target node is divided by the minimum node size, as the vertex positions at the minimum node size.

[0197] According to this configuration, by using a decoded point group that has already been generated for another purpose for processing, it is possible to prevent an increase in the amount of processing.

[0198] Although an example of a method for generating vertices based on a group of decoded points in the Trisoup node size has been described above, the vertices may be generated by a method of interpolating from vertices in the Trisoup node size, as shown in FIG. 12.

[0199] Specifically, for example, first, the approximate surface synthesis unit 2030 calculates the equation of a straight line connecting adjacent vertices based on the result of sorting the vertices in step S1003 or step S1004.

[0200] FIG. 12A shows an example of a case where the Trisoup node is divided into a minimum Trisoup node size, and an example of vertices in the Trisoup node size, similarly to FIG. 11A.

[0201] FIG. 12A shows an example in which straight lines are generated connecting the 0th and 1st vertices, the 1st and 2nd vertices, and the 2nd and 0th vertices in the sorted index.

[0202] The line l connecting two points a = (ax, ay, az) and b = (bx, by, bz) can be expressed, for example, as l = (1-t)a + tb.

[0203] By finding the intersections between this line and the lines of each edge at the minimum Trisoup node size, the points on the edges can be calculated.

[0204] Here, similarly to FIG. 11B, when the calculation target is limited to edges existing on the surface in the Trisoup node size, the calculation can be simplified as follows.

[0205] First, when finding a line connecting two vertices, if both of the vertices are not on the same surface of the Trisoup node, they can be excluded from the calculation.

[0206] In the example of Figure 12A, the 0th and 1st vertices are on the same surface. Similarly, the 2nd and 0th vertices are on the same surface. On the other hand, the 1st and 2nd vertices are not on the same surface, so the line connecting these two points can be excluded from the calculation.

[0207] If two vertices are limited to being on the same surface, the equation of the line connecting the two vertices can be expressed as the equation of a line on a two-dimensional plane.

[0208] For example, since the 0th vertex and the 1st vertex have the same z coordinate, the line connecting the two vertices can be expressed as a line on the xy plane in the form y = ax + b (a and b are constants).

[0209] On this xy plane, when divided into the minimum Trisoup node size, the edge is x = C × 2 (N-1) Or y = D x 2 (N-1)(C and D are constants), so we can add x=C×2 to the equation of the line above. (N-1) Or y = D x 2 (N-1) By substituting, the intersection point can be calculated.

[0210] Both the intersection point calculated as above and the vertex at the Trisoup node size are regarded as vertices after interpolation.

[0211] FIG. 12B shows an example of vertices generated by the method described above.

[0212] As described above, when the node size is not the minimum node size, the approximate surface synthesis unit 2030 may be configured to determine the intersection of a straight line connecting two of the node vertices and an edge when the node is divided by the minimum node size as the vertex position at the minimum node size.

[0213] With this configuration, the intersection point of the two straight lines can be mathematically calculated uniquely, so that the vertex position can be calculated accurately.

[0214] After the generation of the vertices is completed as described above, the process proceeds to step S1001.

[0215] FIG. 13 is a flowchart showing an example of the vertex decoding process used in step S1002.

[0216] As shown in FIG. 13, in step S1301, the approximate surface synthesis unit 2030 determines whether or not the processing at all trisoup_depths has been completed.

[0217] If the processing has been completed for all trisoup_depth, the process proceeds to step S1305, where the processing ends. If the processing has not been completed for all trisoup_depth, the process proceeds to step S1302.

[0218] In step S1302, the approximate surface synthesis unit 2030 checks the number of unique segments belonging to the target Trisoup hierarchy.

[0219] If the number of unique segments is "0", that is, if the target Trisoup hierarchy does not include even one Trisoup node, the approximate surface synthesis unit 2030 proceeds to step S1301 to process the next Trisoup hierarchy.

[0220] If the number of unique segments is greater than "0", the approximate surface synthesis unit 2030 proceeds to step S1303.

[0221] In step S1303, the approximate surface synthesis unit 2030 decodes whether or not each unique segment includes a vertex to be used in the Trisoup process.

[0222] Note that the number of vertices that can exist for each unique segment may be limited to one. In this case, the number of unique segments in which a vertex exists can be interpreted as being equal to the number of vertices.

[0223] After the approximate surface synthesis unit 2030 has decoded the presence or absence of vertices for all unique segments in the target Trisoup hierarchy, the process proceeds to step S1304.

[0224] In step S1304, the approximate surface synthesis unit 2030 decodes position information indicating where on each unique segment the vertex exists, for each unique segment determined in step S1303 to have a vertex.

[0225] FIG. 14 is a flowchart showing an example of the process in step S1304.

[0226] As shown in FIG. 14, in step S1401, the approximate surface synthesis unit 2030 determines whether the processing has been completed for all segments.

[0227] If the process has been completed for all segments, the process proceeds to step S1406, where the process ends. If the process has not been completed for all segments, the process proceeds to step S1402.

[0228] In step S1402, the approximate surface synthesis unit 2030 acquires the direction of the segment. Specifically, the approximate surface synthesis unit 2030 acquires whether the segment is oriented along the x-axis, y-axis, or z-axis.

[0229] For example, such information is obtained by assigning indexes in a predetermined order to edges (=segments) belonging to the node to be decoded and recording the direction of the segments. In this process, each segment is processed in index order, and the direction of the segment can be determined from the record.

[0230] As another method, the approximate surface synthesis unit 2030 can determine which axis direction the vertex is along by storing information on the coordinates of the start point and the coordinates of the end point for each segment. After obtaining the direction of the segment as described above, the process proceeds to step S1403.

[0231] In step S1403, the approximate surface synthesis unit 2030 acquires the size of the segment.

[0232] The size of the segment corresponds to the node size at the depth of the target octree.

[0233] When using only Octree, the size will be the same for all x, y and z axes, whereas when using QtBt, the values ​​can be different for each of the x, y and z axes.

[0234] The approximate surface synthesis unit 2030 obtains the size of the segment based on the node sizes in the x-axis, y-axis, and z-axis directions at the depth and the direction of the segment obtained in step S1402.

[0235] Alternatively, if the approximate surface synthesis unit 2030 holds information on the coordinates of the start point and the coordinates of the end point for each segment, the size of the segment can be obtained by subtracting the coordinate value of the start point from the coordinate value of the end point.

[0236] After acquiring the size of the segment, the approximate surface synthesis unit 2030 proceeds to step S1404.

[0237] In step S1404, the approximate surface synthesis unit 2030 sets the size of the segment acquired in step S1403 as the maximum possible value for the Position of the vertex to be decoded, and then proceeds to step S1405.

[0238] In step S1405, the approximate surface synthesis unit 2030 sets the range of possible values ​​of Position to 0 to "the maximum value set in S1404"-1, and then decodes Position.

[0239] For example, the approximate surface synthesis unit 2030 may L , then max=2 L-1 At this time, for example, Position may be coded with an L-bit equal-length code (the point group decoding device 200 may decode on the premise that it has been coded in this way).

[0240] In addition, the approximate surface synthesis unit 2030 sets the possible value range of Position to 0 to 2. L-1 Then, entropy coding / decoding may be performed using Huffman coding, arithmetic coding, etc. After decoding Position, the approximate surface synthesis unit 2030 proceeds to step S1401.

[0241] As described above, when decoding the vertex positions of Trisoup, the approximate surface synthesis unit 2030 may be configured to set a range of possible values ​​for the vertex positions for each segment according to the node shape, and to decode the vertex positions based on the range of possible values ​​for the vertex positions.

[0242] In addition, the approximate surface synthesis unit 2030 may be configured to set the range of possible values ​​for the vertex positions based on the axial direction of each segment when generating segments from nodes and the node shape, as described above.

[0243] Furthermore, the approximate surface synthesis unit 2030 may be configured to set the range of possible values ​​for the vertex positions based on the start point coordinates and end point coordinates of each segment, as described above.

[0244] With this configuration, Trisoup can be applied even when the node size is a rectangular prism rather than a cube, which is expected to improve coding efficiency.

[0245] (Point cloud encoding device 100) Hereinafter, the point group encoding device 100 according to this embodiment will be described with reference to Fig. 15. Fig. 15 is a diagram showing an example of functional blocks of the point group encoding device 100 according to this embodiment.

[0246] As shown in FIG. 15, the point cloud encoding device 100 includes a coordinate transformation unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information encoding unit 1050, a geometric information reconstruction unit 1060, a color conversion unit 1070, an attribute transfer unit 1080, a RAHT unit 1090, a LoD calculation unit 1100, a lifting unit 1110, an attribute information quantization unit 1120, and an attribute information encoding unit 1130.

[0247] The coordinate conversion unit 1010 is configured to convert the three-dimensional coordinate system of the input point cloud into any different coordinate system. The coordinate conversion may convert the x, y, and z coordinates of the input point cloud into any s, t, and u coordinates by rotating the input point cloud, for example. As a variation of the conversion, the coordinate system of the input point cloud may be used as it is.

[0248] The geometric information quantization unit 1020 is configured to quantize the position information of the input point group after the coordinate transformation and to remove points with overlapping coordinates. When the quantization step size is 1, the position information of the input point group and the position information after quantization match. In other words, when the quantization step size is 1, it is equivalent to the case where quantization is not performed.

[0249] The tree analysis unit 1030 is configured to receive position information of the quantized point group as input, and to generate an occupancy code indicating at which node in the encoding target space a point exists, based on a tree structure described below.

[0250] In this process, the tree analysis unit 1030 is configured to recursively divide the encoding target space into rectangular parallelepipeds to generate a tree structure.

[0251] If a point exists within a certain rectangular parallelepiped, a tree structure can be generated by recursively dividing the rectangular parallelepiped into multiple rectangular parallelepipeds until the rectangular parallelepiped reaches a specified size. Each such rectangular parallelepiped is called a node. Each rectangular parallelepiped generated by dividing a node is called a child node, and the occupancy code is expressed as 0 or 1 to indicate whether or not a point is included in the child node.

[0252] As described above, the tree analysis unit 1030 is configured to generate occupancy codes while recursively dividing nodes until a predetermined size is reached.

[0253] In this embodiment, a method called "Octree" can be used, which recursively performs octree division on the above-mentioned rectangular parallelepiped, always treating it as a cube, and a method called "QtBt" can be used, which performs quadtree division and binary tree division in addition to octree division.

[0254] Here, whether or not to use “QtBt” is transmitted to the point cloud decoding device 200 as control data.

[0255] Alternatively, it may be specified to use predictive coding using an arbitrary tree structure. In this case, the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoding device 200 as control data.

[0256] For example, the tree-structured control data may be configured so as to be decoded according to the procedure described with reference to FIG.

[0257] The approximate surface analyzer 1040 is configured to generate approximate surface information using the tree information generated by the tree analyzer 1030 .

[0258] Approximate surface information is used when, for example, decoding three-dimensional point cloud data of an object, in cases where the point cloud is densely distributed on the object surface, to approximate the area in which the point cloud exists using a small plane, rather than decoding each individual point cloud.

[0259] Specifically, the approximate surface analysis unit 1040 may be configured to generate approximate surface information using, for example, a method called "Trisoup." In addition, when decoding a sparse point cloud acquired by Lidar or the like, this process can be omitted.

[0260] The geometric information encoding unit 1050 is configured to generate a bit stream (geometric information bit stream) by encoding syntax such as the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040. Here, the bit stream may include, for example, the syntax described in Fig. 4 and Fig. 5.

[0261] The encoding process is, for example, a context-adaptive binary arithmetic encoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0262] The geometric information reconstruction unit 1060 is configured to reconstruct geometric information (the coordinate system assumed by the encoding process, i.e., the position information after coordinate transformation in the coordinate transformation unit 1010) of each point of the point cloud data to be encoded, based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040.

[0263] The color conversion unit 1070 is configured to perform color conversion when the input attribute information is color information. The color conversion does not necessarily have to be performed, and the presence or absence of the color conversion process is coded as part of the control data and transmitted to the point cloud decoding device 200.

[0264] The attribute transfer unit 1080 is configured to correct the attribute values ​​based on the position information of the input point cloud, the position information of the point cloud after reconstructing in the geometric information reconstruction unit 1060, and the attribute information after color change in the color conversion unit 1070, so as to minimize distortion of the attribute information.

[0265] The RAHT unit 1090 is configured to receive the attribute information transferred by the attribute transfer unit 1080 and the geometric information generated by the geometric information reconstruction unit 1060 as input, and to generate residual information for each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform). As a specific example of the RAHT process, the method described in the above-mentioned document 2 can be used.

[0266] The LoD calculation unit 1100 is configured to receive the geometric information generated by the geometric information reconstruction unit 1060 as an input and generate a Level of Detail (LoD).

[0267] LoD is information for defining a reference relationship (a referencing point and a referenced point) to realize predictive coding, such as predicting attribute information of a certain point from attribute information of another point and encoding or decoding the prediction residual.

[0268] In other words, LoD is information that defines a hierarchical structure in which each point contained in geometric information is classified into multiple levels, and attributes of points belonging to lower levels are encoded or decoded using attribute information of points belonging to higher levels.

[0269] As a specific method for determining the LoD, for example, the method described in the above-mentioned document 2 may be used.

[0270] The lifting unit 1110 is configured to generate residual information by a lifting process using the LoD generated by the LoD calculation unit 1100 and the attribute information after attribute transfer by the attribute transfer unit 1080.

[0271] As a specific example of the lifting process, the method described in the literature (Text of ISO / IEC 23090-9 DIS Geometry-based PCC, ISO / IEC JTC1 / SC29 / WG11 N19088) may be used.

[0272] The attribute information quantization unit 1120 is configured to quantize the residual information output from the RAHT unit 1090 or the lifting unit 1110. Here, a quantization step size of 1 is equivalent to no quantization being performed.

[0273] The attribute information encoding unit 1130 is configured to perform encoding processing using the quantized residual information and the like output from the attribute information quantization unit 1120 as syntax, and to generate a bit stream related to the attribute information (attribute information bit stream).

[0274] The encoding process is, for example, a context-adaptive binary arithmetic encoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0275] Through the above processing, the point group encoding device 100 is configured to perform encoding processing using position information and attribute information of each point in a point group as input, and to output a geometric information bit stream and an attribute information bit stream.

[0276] Furthermore, the above-mentioned point group encoding device 100 and point group decoding device 200 may be realized as a program that causes a computer to execute each function (each process).

[0277] In each of the above embodiments, the present invention has been described using the point cloud encoding device 100 and the point cloud decoding device 200 as examples, but the present invention is not limited to such examples and can be similarly applied to a point cloud encoding / decoding system having the functions of the point cloud encoding device 100 and the point cloud decoding device 200. [Industrial Applicability]

[0278] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0279] 10...Point cloud processing system 100...Point cloud encoding device 1010... Coordinate conversion section 1020...Geometric information quantization section 1030…Tree analysis section 1040…Approximate surface analysis section 1050...Geometric information encoding unit 1060...Geometric information reconstruction unit 1070…Color conversion section 1080…Attribute transfer section 1090…RAHT Section 1100…LoD calculation section 1110…Lifting section 1120...Attribute information quantization section 1130...Attribute information encoding unit 200...Point cloud decoding device 2010…Geometric Information Decoding Department 2020…Tree synthesis section 2030…Approximate surface synthesis part 2040...Geometric information reconstruction unit 2050…Inverse coordinate conversion section 2060…Attribute information decoding unit 2070...Inverse quantization section 2080…RAHT Division 2090…LoD calculation section 2100…Reverse lifting section 2110…Color inversion unit

Claims

1. A point cloud decoding device, comprising: An approximate surface synthesis unit is provided, The approximate surface synthesis unit includes: Based on the maximum node size and the minimum node size decoded from the control data, a decode process is performed for the vertices at each node size between the maximum and minimum; if the node size is not the minimum node size, generating vertex positions at the minimum node size based on the decoded vertices; A point cloud decoding device characterized in that, when there are multiple vertices on a certain edge generated from nodes of different node sizes adjacent to the certain edge, the vertex generated by the node with the largest node size among the different adjacent nodes is configured to be the vertex position of the certain edge.

2. The point cloud decoding device according to claim 1 , characterized in that the approximate surface synthesis unit is configured to generate vertices only on the surface of the node when generating the vertex positions for the minimum node size.

3. The point cloud decoding device according to claim 1 or 2, characterized in that, when the node size is not the minimum node size, the approximate surface synthesis unit is configured to set, among the points in the decoded point cloud of the node to be decoded, the points on the edges when the node to be decoded is divided by the minimum node size, as the vertex positions at the minimum node size.

4. The point cloud decoding device of claim 1 or 2, characterized in that, when the node size is not the minimum node size, the approximate surface synthesis unit is configured to set the intersection of a straight line connecting two of the vertices of the node to be decoded and an edge when the node to be decoded is divided by the minimum node size as the vertex position at the minimum node size.

5. 1. A point cloud decoding method, comprising: performing a decoding process for vertices at each node size between the maximum and minimum based on the maximum and minimum node sizes decoded from the control data; if the node size is not the minimum node size, generating a vertex position at the minimum node size based on the decoded vertex; A point cloud decoding method characterized by comprising a step of, when there are multiple vertices on a certain edge generated from nodes of different node sizes adjacent to the certain edge, setting the vertex generated by the node with the largest node size among the different adjacent nodes as the vertex position of the certain edge.

6. A program for causing a computer to function as a point group decoding device, The point group decoding device comprises: An approximate surface synthesis unit is provided, The approximate surface synthesis unit includes: Based on the maximum node size and the minimum node size decoded from the control data, a decode process is performed for the vertices at each node size between the maximum and minimum; if the node size is not the minimum node size, generating vertex positions at the minimum node size based on the decoded vertices; A program characterized by being configured such that, when there are multiple vertices on a certain edge generated from nodes of different node sizes adjacent to the certain edge, the vertex generated by the node with the largest node size among the different adjacent nodes is set as the vertex position of the certain edge.

Images