Point group decoding device, point group decoding method and program

Abstract

To facilitate design by regulating constraints regarding size of an initial node when splitting is performed.SOLUTION: A point group decoding device 200 according to the present invention comprises: a geometric information decoding unit 2010 constituted to decode a flag for controlling whether or not "Implicit QtBt" is applied; and an attribute information decoding unit 2060 constituted to decode a flag for controlling whether or not "scalable lifting" is applied, and is constrained so that the "scalable lifting" is not applied when the "Implicit QtBt" is applied.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a point group decoding device, a point group decoding method, and a program. [Background technology]

[0002] Non-Patent Documents 1 and 2 disclose techniques for decoding position information of a point group by recursively performing 8-tree division, 4-tree division, and 2-tree division. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] [G-PCC]Editorial input on G-PCC draft text, ISO / IEC, JTC1 / SC29 / WG11, m51806 [Non-Patent Document 2] G-PCC codec description V5, ISO / IEC, JTC1 / SC29 / WG11, N18891 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the inventors of the present application have found that Non-Patent Documents 1 and 2 have a problem in that no constraints are defined regarding the size of the initial node when performing the above-mentioned division.

[0005] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a point cloud decoding device, a point cloud decoding method, and a program that can facilitate design by making it possible to specify constraints on the size of the initial node when performing division. [Means for solving the problem]

[0006] A first feature of the present invention is summarized as a point cloud decoding device including a geometric information decoding unit configured to decode a syntax that specifies a maximum value of an initial node size.

[0007] A second feature of the present invention is that it is a point cloud decoding device comprising a geometric information reconstruction unit configured to output position information of each point as one-dimensional array data and to arrange the order of the array data so that data of points belonging to the same parent node are adjacent to each other.

[0008] A third feature of the present invention is a point cloud decoding device comprising a geometric information decoding unit configured to decode a flag that controls whether to apply "Implicit QtBt" and an attribute information decoding unit configured to decode a flag that controls whether to apply "Scalable Lifting", and wherein when the "Implicit QtBt" is applied, the "Scalable Lifting" is restricted to not be applied.

[0009] A fourth feature of the present invention is summarized as a point cloud decoding device comprising: a tree synthesis unit configured to decode position data of each point by recursively dividing an initial node; and an LoD calculation unit configured to calculate a Morton Code from coordinates of each point and sort the position information data of each point by referring to a bit of the Morton Code corresponding to a node size in the division process.

[0010] A fifth feature of the present invention is summarized as a point cloud decoding device comprising: a tree synthesis unit configured to decode position data of each point by recursively dividing an initial node; and an LoD calculation unit configured to calculate, for each point, a code that generates bits corresponding to each of the x, y, and z axes for each division process, and to sort the position information data of each point in order of the codes.

[0011] A sixth aspect of the present invention is summarized as a point group decoding method including a step of decoding a syntax that specifies a maximum value of an initial node size.

[0012] A seventh aspect of the present invention is a program for use in a point group decoding device, the program causing a computer to execute a step of decoding a syntax that specifies a maximum value of an initial node size. Effect of the Invention

[0013] According to the present invention, it is possible to provide a point cloud decoding device, a point cloud decoding method, and a program that can facilitate design by making it possible to specify constraints regarding the size of the initial node when performing division. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a point cloud processing system 10 according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of functional blocks of a point group decoding device 200 according to an embodiment. [Diagram 3] 1 is a diagram illustrating an example of the configuration of encoded data (bit stream) received by a geometric information decoding unit 2010 of a point cloud decoding device 200 according to an embodiment. [Figure 4] 1 is an example of a syntax configuration of GPS2011 according to an embodiment. [Diagram 5] 1 is an example of a syntax configuration of GSH2012A / 2012B according to an embodiment. [Figure 6] FIG. 13 is a diagram for explaining the syntax of GSH2012A / 2012B according to an embodiment. [Figure 7] 11 is a diagram for explaining an example of processing by a tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. FIG. [Figure 8] 11 is a diagram for explaining an example of processing by a tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. FIG. [Figure 9]2 is a diagram for explaining an example of a tree structure divided by processing of a tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. FIG. [Figure 10] 13 is a diagram illustrating an example of the configuration of encoded data (bit stream) received by an attribute information decoding unit 2060 of the point group decoding device 200 according to an embodiment. [Figure 11] 1 is an example of a syntax configuration of APS2061 according to an embodiment. [Figure 12] 11 is a flowchart illustrating an example of a processing procedure of a LoD calculation unit 2090 of the point cloud decoding device 200 according to an embodiment. [Figure 13] 11 is a diagram for explaining an example of processing by a LoD calculation unit 2090 of the point cloud decoding device 200 according to an embodiment. FIG. [Figure 14] 11 is a diagram for explaining an example of processing by a LoD calculation unit 2090 of the point cloud decoding device 200 according to an embodiment. FIG. [Figure 15] 11 is a diagram for explaining an example of processing by a LoD calculation unit 2090 of the point cloud decoding device 200 according to an embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0016] (First embodiment) A point cloud processing system 10 according to a first embodiment of the present invention will be described below with reference to Fig. 1 to Fig. 8. Fig. 1 is a diagram showing a point cloud processing system 10 according to the embodiment.

[0017] As shown in FIG. 1, a point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0018] The point cloud encoding device 100 is configured to generate encoded data (bit stream) by encoding an input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bit stream.

[0019] The input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud, such as color information and reflectance of each point.

[0020] Here, such a bit stream may be transmitted from the point group encoding device 100 to the point group decoding device 200 via a transmission path. Also, the bit stream may be stored in a storage medium and then provided from the point group encoding device 100 to the point group decoding device 200.

[0021] (Point Cloud Decoding Device 200) Hereinafter, the point group decoding device 200 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the point group decoding device 200 according to this embodiment.

[0022] As shown in FIG. 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, an LoD calculation unit 2090, an inverse lifting unit 2100, and an inverse color transformation unit 2110.

[0023] The geometric information decoding unit 2010 is configured to receive as input a bit stream relating to geometric information (geometric information bit stream) out of the bit streams output from the point group encoding device 100, and to decode the syntax.

[0024] The decoding process is, for example, a context adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0025] The tree synthesis unit 2020 is configured to receive as input the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating which node in the tree described below the point group exists in, and generate tree information indicating in which area in the space to be decoded the point exists.

[0026] This process divides the space to be decoded into rectangles, determines whether a point exists in each rectangle by referring to the occupancy code, divides the rectangle in which the point exists into multiple rectangles, and then refers to the occupancy code, and recursively repeats this process to generate tree information.

[0027] In this embodiment, a method called "Octree" can be used, which always treats the above-mentioned rectangular parallelepiped as a cube and recursively divides it into octrees, quadtrees, and binary trees, and a method called "Implicit QtBt" can be used, which specifies the initial size of the rectangular parallelepiped by control data and adaptively divides it into octrees, quadtrees, and binary trees based on the size of the rectangular parallelepiped and parameters in the control data. Which method to use is transmitted from the point cloud encoding device 100 as control data.

[0028] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree information synthesis unit 2020.

[0029] Approximate surface information is used when, for example, decoding three-dimensional point cloud data of an object, in cases where the point cloud is densely distributed on the object surface, to approximate the area in which the point cloud exists using a small plane, rather than decoding each individual point cloud.

[0030] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information, for example, by a method called "Trisoup". As a specific process of "Trisoup", for example, the methods described in Non-Patent Documents 1 and 2 can be used. In addition, when decoding sparse point cloud data acquired by Lidar or the like, this process can be omitted.

[0031] The geometric information reconstruction unit 2040 is configured to reconstruct geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point group to be decoded, based on the tree information generated by the tree information synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0032] The inverse coordinate transformation unit 2050 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform the information from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

[0033] The attribute information decoding unit 2060 is configured to receive as input a bit stream relating to attribute information (attribute information bit stream) out of the bit streams output from the point group encoding device 100, and to decode the syntax.

[0034] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0035] Moreover, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

[0036] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and a quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, to generate inverse quantized residual information.

[0037] The dequantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090 according to the features of the point group to be decoded. Control data decoded by the attribute information decoding unit 2060 specifies which unit the dequantized residual information is output to.

[0038] The RAHT unit 2080 is configured to receive the inverse quantized residual information generated by the inverse quantized residual information and the geometric information generated by the geometric information reconstruction unit 2040, and to decode the attribute information of each point using a type of Haar transform (inverse Haar transform in the decoding process) called RAHT (Region Adaptive Hierarchical Transform). As a specific process of RAHT, for example, the methods described in Non-Patent Documents 1 and 2 can be used.

[0039] The LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 as an input and generate a Level of Detail (LoD).

[0040] LoD is information for defining a reference relationship (a referencing point and a referenced point) to realize predictive coding, such as predicting attribute information of a certain point from attribute information of another point and encoding or decoding the prediction residual.

[0041] In other words, LoD is information that defines a hierarchical structure in which each point contained in geometric information is classified into multiple levels, and attributes of points belonging to lower levels are encoded or decoded using attribute information of points belonging to higher levels.

[0042] As a specific method for determining the LoD, for example, the methods described in Non-Patent Documents 1 and 2 may be used. Other examples will be described later.

[0043] The inverse lifting unit 2100 is configured to decode attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse quantized residual information generated by the inverse quantized residual information. As a specific process of the inverse lifting, for example, the methods described in Non-Patent Documents 1 and 2 can be used.

[0044] The inverse color conversion unit 2110 is configured to perform inverse color conversion processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information to be decoded is color information and color conversion has been performed on the point group encoding device 100 side. Whether or not to perform such inverse color conversion processing is determined by the control data decoded by the attribute information decoding unit 2060.

[0045] The point group decoding device 200 is configured to decode and output the attribute information of each point in the point group through the above processing.

[0046] (Geometric Information Decoding Part 2010) The control data decoded by the geometric information decoding unit 2010 will be described below with reference to FIGS.

[0047] FIG. 3 shows an example of the structure of the coded data (bit stream) received by the geometric information decoding unit 2010. As shown in FIG.

[0048] First, the bit stream may include a GPS 2011. The GPS 2011 is also called a geometry parameter set, and is a set of control data related to decoding of geometric information. A specific example will be described later. Each GPS 2011 includes at least GPS ID information for identifying each GPS 2011 when multiple GPS 2011 exist.

[0049] Secondly, the bit stream may include GSH2012A / 2012B. GSH2012A / 2012B is also called a geometry slice header and is a set of control data corresponding to a slice described later. A specific example will be described later. GSH2012A / 2012B includes at least GPS id information for specifying the GPS2011 corresponding to each GSH2012A / 2012B.

[0050] Thirdly, the bit stream may include slice data 2013A / 2013B following the GSH 2012A / 2012B. The slice data 2013A / 2013B includes data that encodes geometric information. An example of the slice data 2013A / 2013B is an occupancy code, which will be described later.

[0051] As described above, the bit stream is configured such that each slice data 2013A / 2013B corresponds to one GSH 2012A / 2012B and one GPS 2011.

[0052] As described above, since the GPS ID information is used to specify which GPS 2011 to refer to in the GSH 2012A / 2012B, a common GPS 2011 can be used for a plurality of slice data 2013A / 2013B.

[0053] In other words, it is not necessary to transmit the GPS 2011 for each slice. For example, as shown in Fig. 3, the bit stream may be configured such that the GPS 2011 is not coded immediately before the GSH 2012B and the slice data 2013B.

[0054] The configuration in Fig. 3 is merely an example. As long as the GSH 2012A / 2012B and the GPS 2011 correspond to each slice data 2013A / 2013B, elements other than those described above may be added as components of the bit stream. For example, the bit stream may include a sequence parameter set (SPS). Similarly, the bit stream may be shaped into a different configuration from that in Fig. 3 when transmitted. Furthermore, the bit stream may be combined with a bit stream decoded by an attribute information decoding unit 2060 (described later) and transmitted as a single bit stream.

[0055] FIG. 4 is an example of the syntax configuration of GPS2011.

[0056] The GPS 2011 may include GPS ID information (gps_geom_parameter_set_id) for identifying each GPS 2011.

[0057] GPS2011 may include a syntax (gps_max_log2_node_size) that specifies the maximum node size (maximum initial node size) of a tree described later. gps_max_log2_node_size may be specified to always take a value of "0" or more. gps_max_log2_node_size may be coded, for example, with an unsigned zeroth-order exponential Golomb code.

[0058] The Descriptor column in Fig. 4 indicates how each syntax is coded. ue(v) indicates an unsigned zeroth-order exponential Golomb code, and u(1) indicates a 1-bit flag.

[0059] GPS2011 includes gps_max_log2_node_size, which specifies the maximum node size, making it possible to specify the maximum value that can be taken by the syntax described below.

[0060] In addition, by including gps_max_log2_node_size in GPS2011, the maximum value that can be set for gps_max_log2_node_size can be defined within rules called profiles or levels, making it easy to specify the maximum node size that should be supported depending on performance and applications.

[0061] The GPS 2011 may include a flag (gps_implicit_geom_partition_flag) for controlling whether or not to perform "Implicit QtBt" in the tree synthesis unit 2020. For example, when the value of gps_implicit_geom_partition_flag is "1", it may be defined that "Implicit QtBt" is performed, and when the value of gps_implicit_geom_partition_flag is "0", it may be defined that "Octree" is performed, which performs only octree division with the tree nodes always being cubes.

[0062] If the value of gps_implicit_geom_partition_flag is "1", GPS2011 may additionally include control data related to "Implicit QtBt".

[0063] The GPS2011 may include, as control data related to "Implicit QtBt", a maximum value (gps_max_diff_log2_node_size) of the difference between the maximum and minimum values ​​of the initial node size in each of the x-axis, y-axis, and z-axis directions.

[0064] When the difference between the maximum and minimum values ​​of the initial node size in each of the x-axis, y-axis, and z-axis directions (the value of gps_max_diff_log2_node_size) is smallest, that is, when the initial node sizes in each of the x-axis, y-axis, and z-axis directions are equal, it means that the shape of the node is a cube.

[0065] Conversely, as the difference between the maximum and minimum values ​​of the initial node size in each of the x-axis, y-axis, and z-axis directions (the value of gps_max_diff_log2_node_size) increases, the initial shape of the node becomes an elongated or flat rectangular parallelepiped shape.

[0066] The value of gps_max_diff_log2_node_size may be specified to always be a value equal to or greater than "0." Also, the value of gps_max_diff_log2_node_size may be specified to always be a value equal to or less than the value of gps_max_log2_node_size.

[0067] By providing gps_max_diff_log2_node_size as control data in GPS2011, it is possible to define minimum and maximum values ​​of syntax related to initial size data in each of the x-axis, y-axis, and z-axis directions of a node, which will be described later.

[0068] Furthermore, by providing gps_max_diff_log2_node_size to GPS2011, cases in which the initial node shape becomes extremely elongated or extremely flat can be excluded, making it easier to design the point cloud decoding device 200.

[0069] GPS2011 may include a value (gps_max_num_implicit_qtbt_before_ot) corresponding to K described later and a value (gps_min_size_implicit_qtbt) corresponding to M described later as control data related to "Implicit QtBt".

[0070] The value of gps_max_num_implicit_qtbt_before_ot may be specified to always be a value equal to or greater than "0." In addition, the value of gps_max_num_implicit_qtbt_before_ot may be specified to always be a value equal to or less than the value of gps_max_log2_node_size.

[0071] The value of gps_min_size_implicit_qtbt may be specified to always be a value equal to or greater than "0." In addition, the value of gps_min_size_implicit_qtbt may be specified to always be a value equal to or less than the value of gps_max_log2_node_size.

[0072] FIG. 5 is an example of the syntax configuration of GSH.

[0073] GHS2012A / 2012B may include a syntax (gsh_geometry_parameter_set_id) for specifying GPS2011 corresponding to the GHS2012A / 2012B.

[0074] GHS2012A / 2012B may additionally include control data related to "Implicit QtBt" if the value of gps_implicit_geom_partition_flag is "1" in GPS2011 corresponding to the GHS2012A / 2012B.

[0075] As control data related to "Implicit QtBt", GSH2012A / 2012B may include a flag (gsh_implicit_geom_partition_flag) that controls whether to apply "Implicit QtBt" or "Octree" when decoding slice data 2013A / 2013B corresponding to the GSH2012A / 2012B.

[0076] For example, when the value of gsh_implicit_geom_partition_flag is "1", it may be defined that "Implicit QtBt" is applied, and when the value of gsh_implicit_geom_partition_flag is "0", it may be defined that "Octree" is applied.

[0077] If GSH2012A / 2012B does not include gsh_implicit_geom_partition_flag, the geometric information decoding unit 2010 may regard the value of gsh_implicit_geom_partition_flag as "0".

[0078] By providing gsh_implicit_geom_partition_flag in GSH2012A / 2012B, it becomes possible to control the applicability of "Implicit QtBt" on a slice-by-slice basis.

[0079] When the value of gsh_implicit_geom_partition_flag is “1”, GSH2012A / 2012B may include syntax (gsh_log2_max_nodesize_x) that indicates the initial node size in the x-axis direction.

[0080] The value of gsh_log2_max_nodesize_x may be specified to always be a value equal to or greater than "0." Also, the value of gsh_log2_max_nodesize_x may be specified to always be a value equal to or less than gps_max_log2_node_size.

[0081] As shown in FIG. 6, the initial node size in the x-axis direction may be represented by a variable MaxNodeSizeXLog2, which may be defined as MaxNodeSizeXLog2=gsh_log2_max_nodesize_x.

[0082] When the value of gsh_implicit_geom_partition_flag is "1", GPS2011 may include the value (gsh_log2_max_nodesize_y_minus_x) obtained by subtracting the initial node size in the x-axis direction (MaxNodeSizeXLog2) from the initial node size in the y-axis direction as a syntax. gsh_log2_max_nodesize_y_minus_x may be coded using signed zeroth order exponential Golomb (se(v)).

[0083] The value of gsh_log2_max_nodesize_y_minus_x may be specified to be greater than or equal to -gps_max_diff_log2_node_size and less than or equal to gps_max_diff_log2_node_size.

[0084] In other words, the absolute value of gsh_log2_max_nodesize_y_minus_x may be specified to be less than or equal to gps_max_diff_log2_node_size.

[0085] Also, it may be specified that the initial node size in the y-axis direction (=MaxNodeSizeXLog2+gsh_log2_max_nodesize_y_minus_x) is equal to or greater than "0" and equal to or less than gps_max_log2_node_size.

[0086] In addition, by combining the above provisions, it may be specified that the value of gsh_log2_max_nodesize_y_minus_x is greater than or equal to Max(-MaxNodeSizeXLog2,-gps_max_diff_log2_node_size) and less than or equal to Min(gps_max_log2_node_size-MaxNodeSizeXLog2,gps_max_diff_log2_node_size).

[0087] Note that Max(a, b) is a function that returns the maximum value of two arguments a and b, and Min(a, b) is a function that returns the minimum value of two arguments a and b.

[0088] As shown in FIG. 6, the initial node size in the y-axis direction may be represented by a variable MaxNodeSizeYLog2, which may be defined as MaxNodeSizeYLog2=MaxNodeSizeXLog2+gsh_log2_max_nodesize_y_minus_x.

[0089] When the value of gsh_implicit_geom_partition_flag is "1", GPS2011 may include the value (gsh_log2_max_nodesize_z_minus_y) obtained by subtracting the initial node size in the y-axis direction (MaxNodeSizeYLog2) from the initial node size in the z-axis direction as syntax.

[0090] The value of gsh_log2_max_nodesize_z_minus_y may be specified to be greater than or equal to -gps_max_diff_log2_node_size and less than or equal to gps_max_diff_log2_node_size.

[0091] In addition, it may be specified that the difference between the initial node size in the z-axis direction and the initial node size in the x-axis direction (=gsh_log2_max_nodesize_y_minus_x+gsh_log2_max_nodesize_z_minus_y) is equal to or greater than -gps_max_diff_log2_node_size and equal to or less than gps_max_diff_log2_node_size.

[0092] In other words, it may be specified that the absolute value of gsh_log2_max_nodesize_z_minus_y is less than or equal to gps_max_diff_log2_node_size, and the absolute value of gsh_log2_max_nodesize_y_minus_x+gsh_log2_max_nodesize_z_minus_y is less than or equal to gps_max_diff_log2_node_size.

[0093] Also, it may be specified that the initial node size in the z-axis direction (=MaxNodeSizeYLog2+gsh_log2_max_nodesize_z_minus_y) must be equal to or greater than "0" and equal to or less than gps_max_log2_node_size.

[0094] In addition, by combining the above provisions, it may be specified that the value of gsh_log2_max_nodesize_z_minus_y is greater than or equal to Max(-MaxNodeSizeYLog2, Max(-gps_max_diff_log2_node_size,-gps_max_diff_log2_node_size-gsh_log2_max_nodesize_y_minus_x)) and less than or equal to Min(gps_max_diff_log2_node_size-MaxNodeSizeYLog2, Min(gps_max_diff_log2_node_size-gsh_log2_max_nodesize_y_minus_x,gps_max_diff_log2_node_size)).

[0095] As shown in FIG. 6, the initial node size in the z-axis direction may be represented by a variable MaxNodeSizeZLog2, which may be defined as MaxNodeSizeZLog2=MaxNodeSizeYLog2+gsh_log2_max_nodesize_z_minus_y.

[0096] If the value of gsh_log2_max_nodesize_z_minus_y is "0", GPS2011 may include syntax (gsh_log2_max_nodesize) that indicates the initial node size.

[0097] The value of gsh_log2_max_nodesize may be specified to be greater than or equal to "0" and less than or equal to gps_max_log2_node_size.

[0098] When the value of gsh_implicit_geom_partition_flag is "0", only cubes are assumed as node shapes, so the initial node size in all directions of the x-axis, y-axis, and z-axis can be specified by gsh_log2_max_size. Such initial node size may be represented by the variable MaxNodeSizeLog2, and defined as MaxNodeSizeLog2=gsh_log2_max_nodesize.

[0099] In the above, an example was shown in which GSH2012A / 2012B has gsh_implicit_geom_partition_flag, but even if gsh_implicit_geom_partition_flag does not exist, processing similar to that described above is possible.

[0100] If gsh_implicit_geom_partition_flag does not exist, the geometric information decoding unit 2010 can decode gsh_log2_max_nodesize_x, gsh_log2_max_nodesize_y_minus_x, and gsh_log2_max_nodesize_z_minus_y when the value of gps_implicit_geom_partition_flag is “1”, and can decode gsh_log2_max_nodesize when the value of gps_implicit_geom_partition_flag is “0”.

[0101] (Tree Synthesis Department 2020) An example of the processing of the tree merging unit 2020 will be described with reference to FIGS.

[0102] FIG. 7 is a diagram for explaining processing when "Octree" is applied, that is, when the value of gps_implicit_geom_partition_flag is "0" or when the value of gsh_implicit_geom_partition_flag is "0."

[0103] When "Octree" is applied, first, the tree synthesis unit 2020 specifies the initial node size as shown in Fig. 7 using MaxNodeSizeLog2 decoded by the geometric information decoding unit 2010. As described above, in the case of "Octree", the node shape is a cube.

[0104] Secondly, the tree merge unit 2020 divides the initial node in half along the x-axis, y-axis and z-axis directions, and divides it into eight child nodes. In the following explanation, for convenience, each child node is labeled with a to h.

[0105] Thirdly, the tree synthesis unit 2020 refers to the occupancy code decoded by the geometric information decoding unit 2010 .

[0106] The first 8 bits of the occupancy code indicate whether the point to be decoded is contained within each of the above-mentioned child nodes a to h. When the value of the occupancy code is "1", it means that the point is present within the child node, and when the value of the occupancy code is "0", it means that the point is not present within the child node. In the example of FIG. 7, it means that the point is present only within child nodes b and h.

[0107] Fourth, the tree merging unit 2020 again divides the child nodes b and h, which have points inside, in half along the x-axis, y-axis, and z-axis directions, respectively, into eight child nodes.

[0108] Fifth, the tree synthesis unit 2020 again refers to the next occupancy code and determines whether a point exists inside each child node.

[0109] Thereafter, the tree synthesis unit 2020 recursively repeats these processes to generate tree information indicating in which area a point exists within the space defined by the initial node.

[0110] FIG. 8 is a diagram for explaining the processing when "Implicit QtBt" is applied, that is, when the value of gps_implicit_geom_partition_flag is "1" or when the value of gsh_implicit_geom_partition_flag is "1."

[0111] When applying "Implicit QtBt", the tree synthesis unit 2020: The initial node size is defined using MaxNodeSizeXLog2, MaxNodeSizeYLog2, and MaxNodeSizeZLog2 decoded by the geometric information decoding unit 2010, as shown in Fig. 8. As described above, when "Implicit QtBt" is applied, the node shape becomes a rectangular parallelepiped (it goes without saying that a cube is also a type of rectangular parallelepiped).

[0112] When applying "Implicit QtBt", the basic process is the same as when applying "Octree". When applying "Octree", the node is always divided into eight child nodes, but when applying "Implicit QtBt", the node may be divided into four or two child nodes.

[0113] FIG. 8(a) shows an example in which a node is divided into four child nodes, and FIG. 8(b) shows an example in which a node is divided into two child nodes.

[0114] As shown in Figure 8(a), when a node is divided into four child nodes, the corresponding occupancy code is 4 bits. Similarly, when a node is divided into two child nodes, as shown in Figure 8(b), the corresponding occupancy code is 2 bits.

[0115] Whether a node is to be divided into 8, 4 or 2 (whether to perform 8-tree division, 4-tree division or 2-tree division) is implicitly determined using the number of divisions from the initial node, the size of the node in the x-axis direction, y-axis direction and z-axis direction, and the values ​​of the control data (K and M) decoded by the geometric information decoding unit 2010. As a specific method of determination, the methods described in Non-Patent Documents 1 and 2 can be used.

[0116] Here, K is a parameter that specifies the maximum number of times quadtree division or binary tree division is performed before the first octree division, and M is a parameter that specifies the minimum node size for quadtree division and binary tree division.

[0117] As described above, by recursively dividing the data until all node sizes are 1x1x1, the positions of each point that makes up the point cloud (each of which corresponds to a 1x1x1 node) can be decoded.

[0118] When scalable lifting, which will be described later, is applied (when lifting_scalability_enabled_flag=1), the decoding process described above may be controlled by the value of a parameter skipOctreeLayers that is determined by the user prior to the operation of the decoding device.

[0119] For example, FIG. 9 shows an example of a tree structure in which all node sizes become 1×1×1 by repeating division three times from the initial node. Here, when the value of skipOctreeLayers is "0", division is performed three times for each node, as described above. Next, for example, when skipOctreeLayers=2, the lower two division processes are omitted. In the case of FIG. 9, this corresponds to omitting the division process for the layers indicated as division times 2 and 3, so the decoding process ends when each node has been divided once.

[0120] The position information of each point decoded as described above is output to the RAHT unit 2080 and the LoD calculation unit 2090 via the approximate surface synthesis unit 2030 and the geometric information reconstruction unit 2040. At this time, the position information of each point may be transmitted as data stored in a one-dimensional array, and a constraint may be defined as the order of such data, such that data of points belonging to the same parent node are arranged adjacent to each other.

[0121] In other words, the geometric information reconstruction unit 2040 may output the position information of each point as one-dimensional array data, with the order of the array data being such that data for points belonging to the same parent node are adjacent to each other.

[0122] For example, in the example of Fig. 9, this means that point data is stored in alphabetical order of abcd...pqr. In this order, for example, when a node in a layer with a division count of 2 is considered as a parent node, nodes a and b, nodes c and d, nodes e and f, etc. are points that belong to the same parent node. Also, when a node in a layer with a division count of 1 is considered as a parent node, nodes a to f, nodes g to i, nodes j to l, and nodes m to r each belong to the same parent node.

[0123] Similarly, the order of the array data described above can be restated as a constraint that the points are arranged in the order they would be if they were processed by breadth-first search.

[0124] (Attribute information decoding unit 2060) The control data decoded by the attribute information decoding unit 2060 will be described below with reference to FIGS.

[0125] FIG. 10 shows an example of the structure of the coded data (bit stream) received by the geometric information decoding unit 2060. As shown in FIG.

[0126] First, the bit stream may include APS2061. APS2061 is also called an attribute parameter set, and is a set of control data related to the decoding of attribute information. A specific example will be described later. Each APS2061 includes at least APS id information for identifying each APS2061 when multiple APS2061 exist.

[0127] Secondly, the bitstream may include ASH2062A / 2062B. ASH2062A / 2062B is also called an attribute slice header and is a set of control data corresponding to slices described later. A specific example will be described later. ASH2062A / 2062B includes at least APS id information for specifying APS2061 corresponding to each ASH2062A / 2062B.

[0128] Thirdly, the bit stream may include slice data 2063A / 2063B following ASH 2062A / 2062B. The slice data 2063A / 2063B includes data in which attribute information is encoded.

[0129] As described above, the bit stream is configured such that each slice data 2063A / 2063B corresponds to one ASH 2062A / 2062B and one APS 2061.

[0130] As described above, in ASH2062A / 2062B, which APS2061 to refer to is specified by APS id information, so that a common APS2061 can be used for a plurality of slice data 2063A / 2063B.

[0131] In other words, it is not necessary to transmit the APS 2061 for each slice. For example, as shown in Fig. 10, the bit stream may be configured such that the APS 2061 is not coded immediately before the ASH 2062B and slice data 2063B.

[0132] Note that the configuration in Fig. 10 is merely an example. As long as the ASH2062A / 2062B and the APS2061 correspond to each slice data 2063A / 2063B, elements other than those described above may be added as components of the bit stream. For example, the bit stream may include a sequence parameter set (SPS).

[0133] Similarly, when transmitting, the data may be shaped into a configuration different from that shown in FIG. 10. Furthermore, the data may be combined with the bit stream decoded by the geometric information decoding unit 2010 and transmitted as a single bit stream. For example, the slice data 2013A and 2063A, and the slice data 2013B and 2063B may each be treated as a single slice data, and GSH2012A and ASH2062A, and GSH2012B and ASH2062B may each be arranged immediately before each slice. In this case, GPS2011 and APS2061 may be arranged prior to each GSH and ASH.

[0134] FIG. 11 is an example of the syntax configuration of APS2061.

[0135] The APS 2061 may include APS id information (aps_attr_parameter_set_id) for identifying each APS 2061.

[0136] The APS 2061 may include information (attr_coding_type) indicating a decoding method of the attribute information. For example, when the value of attr_coding_type is "0", the inverse lifting unit 2100 performs variable weighted lifting prediction, when the value of attr_coding_type is "1", the RAHT unit 2080 performs RAHT, and when the value of attr_coding_type is "2", the inverse lifting unit 2100 performs lifting prediction with a fixed weight.

[0137] The APS 2061 may include a flag (lifting_scalability_enabled_flag) indicating whether to apply scalable lifting when the value of attr_coding_type is '2', that is, when the inverse lifting unit 2100 performs lifting prediction with fixed weights.

[0138] When the value of lifting_scalability_enabled_flag is "0", it may be specified that scalable lifting is not applied, and when the value of lifting_scalability_enabled_flag is "1", it may be specified that scalable lifting is applied.

[0139] Furthermore, when the value of lifting_scalability_enabled_flag is "1", that is, when scalable lifting is applied, it may be specified that skipOctreeLayers is acquired as an external parameter at run time of the image decoding device 200, as described above.

[0140] In addition, when the above-mentioned "Implicit QtBt" is applied, i.e., when the value of gps_implicit_geom_partition_flag is "1", it may be specified that scalable lifting is not applied, i.e., the value of lifting_scalability_enabled_flag is always "0".

[0141] In other words, it is provided with a geometric information decoding unit 2010 that decodes a flag that controls whether to apply "Implicit QtBt", and an attribute information decoding unit 2060 that decodes a flag that controls whether to apply "scalable lifting", and when "Implicit QtBt" is applied, "scalable lifting" may be restricted not to be applied.

[0142] In this way, by restricting the application of "Implicit QtBt" and scalable lifting exclusively, the specification can be simplified by reducing the number of combinations of functions, making implementation easier.

[0143] For example, by restricting the application of scalable lifting to always apply "Octree", the node shape in scalable lifting processing can be performed under the assumption that octree division is always performed at each division. This simplifies the specifications compared to the case where the node shape is a rectangular prism and the division types allowed are quadtree and binary tree division in addition to octree division.

[0144] (LoD calculation unit 2090) An example of the processing contents of the LoD calculation unit 2090 will be described below with reference to FIGS.

[0145] FIG. 12 is an example of a flowchart of the LoD calculation unit 2090.

[0146] In step S1201, the LoD calculation unit 2090 performs a process of rearranging (sorting) the position information data of each point in the point cloud output from the geometric information reconstruction unit 2040.

[0147] Here, as described above, in the output of the tree synthesis unit 2020, if the position information data of each point is arranged in a one-dimensional array and the order is constrained so that the position information data of points belonging to the same parent node are always arranged adjacent to each other, or if the points are constrained to be arranged in the order in which they are processed by breadth-first search, then this step may be omitted and the process may proceed directly to step S1202.

[0148] When there are no constraints as described above, sorting is performed in the following manner. An example of the sorting method will be described with reference to FIGS.

[0149] Fig. 13(a) shows an example of a partitioning result when the initial node size is (x, y) = (4, 8), Fig. 13(b) shows an example of a partitioning tree corresponding to Fig. 13(a), and Fig. 13(c) shows an example of a Morton Code and a Partition-based Code calculated from the coordinates in Fig. 13(a). For simplicity, the following explanation will be given in two dimensions, the x-axis and the y-axis, but the following explanation can be easily extended to three dimensions (x, y, z) by adding information about the z-axis in the same way as the x-axis and y-axis.

[0150] Figure 13(a) shows an example in which the initial node size is (x, y) = (4, 8), and in the first division, the node is quadtree-divided into four child nodes of size (2, 4), in the second division, each of the nodes of size (2, 4) is quadtree-divided into four child nodes of size (1, 2), and in the third division, each of the nodes of size (1, 2) is bitree-divided into two child nodes of size (1, 1).

[0151] Fig. 13(b) corresponds to the number of nodes for which at least one point of location information data exists for each division number described above. In other words, Fig. 13(b) shows the number of nodes for which the occupancy code value is 1.

[0152] Moreover, the alphabets a to r at the bottom of FIG. 13(b) correspond to the nodes a to r of 1×1 size in FIG. 13(a).

[0153] In FIG. 13(a), nodes labeled with a to r have position information data, and other nodes do not have position information data, that is, no point data exists.

[0154] In the following, the coordinates of each node are the points in the area occupied by each node that have the smallest x and y coordinates. In the example of Fig. 13(a), the origin is at the top left of Fig. 13(a), so the coordinates of each node are the coordinates of the top left corner of each node. For example, the coordinates of node p are (x, y) = (3, 5).

[0155] The LoD calculation unit 2090 calculates a Morton Code or a Partition-based Code prior to sorting.

[0156] First, the case where sorting is performed using the Morton Code will be described.

[0157] In the case of a three-dimensional space, the Morton Code is calculated based on the coordinates of each node using the following formula (1).

[0158]

number

[0159] Also, in the case of a two-dimensional space as shown in FIG. 13, the result is as follows.

[0160]

number

[0161]

number

[0162]

number

[0163] FIG. 13(c) shows an example of calculating coordinates from the Morton Code when (x, y)=(3, 5).

[0164] As described above, the LoD calculation unit 2090 calculates the Morton Code for each node, and then performs a sorting process.

[0165] An example of sorting processing when using Morton Code will be described with reference to FIG.

[0166] The "initial" column on the far left of Figure 14 shows the nodes randomly arranged. The six-digit numbers to the right of the alphabets that label each node are the Morton Codes of each node. These Morton Codes are calculated based on the coordinates in Figure 13(a).

[0167] Thereafter, the LoD calculation unit 2090 sorts the data in the "initial" column so that the position information data of points belonging to the same parent node are always adjacent to each other. In other words, the LoD calculation unit 2090 sorts the data in alphabetical order, as in the "node size = (2, 4)" column.

[0168] In the case of Fig. 13(a), both quad-tree division and binary-tree division are applied by applying "Implicit QtBt". In this case, if sorting is done in ascending order of Morton Code, as in the "Morton Code order" column in Fig. 14, there may be cases where the position information data of points belonging to the same parent node are not adjacent. For example, node a belongs to the same parent node as node b, but when sorted in Morton Code order, node a and node c end up being adjacent to each other.

[0169] Below, a method of sorting by applying the concept of radix sorting so that position information data of points belonging to the same parent node are always adjacent to each other will be described.

[0170] First, consider the case where a node is divided into the smallest size. In the example of Figure 13, consider the situation when the division count is 3. In this case, the node size is (x, y) = (1, 1). Therefore, we focus on the digits corresponding to x = 1 and y = 1 in each Morton Code. x = 1 corresponds to the lowest bit of the Morton Code, and y = 1 corresponds to the second lowest bit. In other words, the part enclosed in a box in the "node size = (1, 1)" column in Figure 14 corresponds to x = 1 and y = 1.

[0171] Focusing only on the part enclosed in the square above, the data in the "initial" column is sorted to produce the "node size = (1, 1)" column. As can be seen from Figure 14, the data is sorted in the order of "yx" to be "00", "01", "10", and "11". Note that the order between nodes with the same two-bit pattern in question preserves the order in the "initial" column.

[0172] Secondly, consider the situation when the division count is 2. In this case, the node size is (x, y) = (1, 2). Therefore, we focus on the digits corresponding to x = 1 and y = 2 in each Morton Code. The digit that corresponds to x = 1 is the lowest bit of the Morton Code, and the digit that corresponds to y = 1 is the fourth bit from the lowest. Focusing on the above two bits, the data in the "node size = (1, 1)" column is sorted to obtain the "node size = (1, 2)" column.

[0173] Thirdly, similarly, considering the situation where the division number is 1, by focusing on the digits corresponding to x=2 and y=4 in each Morton Code, the data in the column "node size = (1, 2)" is sorted, resulting in the column "node size = (2, 4)".

[0174] As described above, by sorting from the lowest division layer to the higher division layers in sequence while paying attention only to the node size at each division count and the digits in the Morton Code corresponding to the node size, it is possible to sort so that the position information data of points belonging to the same parent node are always adjacent to each other. Although the above has been explained using an example of a two-dimensional plane of the x-axis and y-axis, sorting can also be performed in the three-dimensional case by additionally considering the z-axis data in each process.

[0175] When applying scalable lifting, a similar sorting can be achieved by starting from the state where the node is divided into the smallest size, and then considering the node size in the reverse order of the division process.

[0176] For example, when the value of the variable skipOctreeLayers is “1”, in the example of Figure 13, the node size is (x, y) = (1, 2), so the process starts by sorting the two bits enclosed in a square in the “node size = (1, 2)” column shown in Figure 14, and then repeats the sorting process up to the top split layer, that is, in the example of Figure 14, until the sorting described in the “node size = (2, 4)” column is completed.

[0177] This concludes the explanation of the sorting method when using the Morton Code.

[0178] That is, the LoD calculation unit 2090 calculates a Morton Code from the coordinates of each point, and sorts the position information data of each point by referring to the bit of the Morton Code that corresponds to the node size in the above-mentioned division process.

[0179] As described above, the Morton Code can be calculated only from the coordinates of each node. In other words, it can be calculated without depending on how the initial node was divided (in which order the 8-tree, 4-tree, or 2-tree division was performed).

[0180] In addition, in the case of "Octree", that is, when the node size in three-dimensional space is always a cube and division is always performed using octree division, by sorting the position data in Morton Code order (for example, from smallest to largest), it is possible to arrange the position information data of points belonging to the same parent node so that they are always adjacent to each other.

[0181] In the case of two-dimensional space, if the node size is always square and the division is always performed using quadtree division, by sorting the position data in Morton Code order (for example, from smallest to largest), it is possible to arrange the position information data of points belonging to the same parent node so that they are always adjacent to each other, as described above.

[0182] Next, we will explain the sorting method when using the Partition-based Code shown in Fig. 13(c). The Partition-based Code treats two bits in the order of "yx" in the case of a two-dimensional plane, and three bits in the order of "zyx" in the case of a three-dimensional space as a group, and indicates which space the coordinates of the corresponding point belong to after each division with each bit ("0" if the coordinate belongs to the smaller space, and "1" if it belongs to the larger space).

[0183] For example, in the example in Fig. 13, after the first division (quadtree division), all of nodes m, o, and p belong to the child node of size (x, y) = (2, 4) in Fig. 13(c). Therefore, the first two bits of the partition-based code (corresponding to the first division) are "11" because both x and y are in the space with the larger coordinates.

[0184] The next two bits of the partition-based code (the middle two bits of the six bits) correspond to the second division (quadtree division). After the second division, among the nodes of size (x, y) = (2, 4) in Figure 13(c), node m belongs to the space with the smaller coordinates for both x and y, so it becomes "00". Nodes o and p have the larger coordinate for x and the smaller coordinate for y, so they become "01".

[0185] The next two bits of the partition-based code (the lowest two bits out of six bits) correspond to the third division (binary tree division). The third division is a division in the y direction only. Since node m and node o each belong to the smaller y coordinates after division, the second lowest bit is "0". The lowest bit corresponds to the x axis, but since there is no division in the x axis direction in the third division, it is always set to "0". Alternatively, bits can be omitted to reduce the number of digits.

[0186] As a result of the above, the lowest two bits of the partition-based codes of nodes m and o are “00 (or only “0”).” Similarly, the lowest two bits of node p are “10 (or only “1”).”

[0187] In other words, the above procedure is equivalent to outputting "1" if the x-axis component of the coordinate for which the Partition-based Code is to be calculated is X, and if the value of X in a certain partition is equal to or greater than "the x-coordinate of the parent node + the size of the child node in the x-axis direction", and outputting "0" otherwise, and similarly executing this for the y-axis and z-axis (in the case of three dimensions). Here, the size of the child node is equal to 1 / 2 the size of the parent node if the partition involves partitioning in the axis direction, and is equal to the size of the parent node if no such partition is performed.

[0188] In other words, for each division process, if the coordinates of a certain point are greater than or equal to the coordinates of the parent node in that division plus the size of the child node in that division, then "1" is output for the x, y, and z axes; otherwise, "0" is output.

[0189] 13(c) shows an example of the case where the partition-based code calculated as described above is inversely converted into coordinates. For example, the coordinates of node p can be inversely converted from the partition-based code as follows.

[0190] First, since the upper two bits correspond to the first division, the node size after the first division (x, y) = (2, 4) is used to set x = 1 × 2 and y = 1 × 4, respectively.

[0191] Secondly, since the two middle bits correspond to the second division, the node size is set to (x, y) = (1, 2), and taking into account the previous result, x = 1 x 2 + 1 x 1 and y = 1 x 4 + 0 x 1. Similarly, for the two lowest bits, taking into account the node size at the third division, the last can be inversely transformed to (x, y) = (3, 5) as shown in Figure 13(c).

[0192] As described above, for each split process, the value obtained by multiplying the bit corresponding to the split ('0' or '1') by the node size after the split (the child node size when viewed from before the split) is calculated for all split processes, and then the sum is taken for each of the x, y, and z axes (in the case of three dimensions), and the coordinates can be inversely converted from the code calculated as described above.

[0193] That is, for each point, the LoD calculation unit 2090 calculates a code that generates bits corresponding to the x, y, and z axes (3 bits in total) (in the case of a three-dimensional space) for each division process, and sorts the position information data of each point in the order of the above-mentioned codes.

[0194] As described above, by calculating the partition-based code that generates bits corresponding to each coordinate axis for each division, the position information data of each point can be arranged so that the position information data of points belonging to the same parent node are always adjacent to each other when the position information data of each point is sorted in the order of the partition-based code. That is, in the example of FIG. 13, the position information data can be sorted in alphabetical order from a to r.

[0195] Furthermore, the above-described Partition-based Code is completely identical to the Morton Code when "Octree" is applied.

[0196] Furthermore, the partition-based code is merely a name given for the sake of convenience, and a code calculated in a similar manner may have a different name. For example, a code that always generates a bit (total of 3 bits) corresponding to each of the x, y, and z axes for each division may be used. Also, for example, a code that is defined to output "1" for each of the x, y, and z axes if the coordinates of a certain point are equal to or greater than the coordinates of the parent node in the division plus the size of the child node in the division, and "0" otherwise may be output. Also, for example, a code that is defined so that coordinates can be inversely converted from the code by performing a calculation taking into account the node size corresponding to the division of the bit.

[0197] As described above, after the LoD calculation unit 2090 calculates the Morton Code or the Partition-based Code and performs the corresponding sorting process, the process proceeds to step S1202.

[0198] In step S1202, the LoD calculation unit 2090 calculates the LoD for each piece of position information of each point.

[0199] The LoD calculation process when scalable lifting is applied will be described below. The LoD calculation method when scalable lifting is not applied can be realized by the method described in Non-Patent Document 1 or Non-Patent Document 2, for example.

[0200] When applying scalable lifting, the LoD calculation process involves sending only one point from among the points belonging to the same parent node to the higher LoD, starting from the lowest LoD (corresponding to a division count of 3 in the example of Figure 13), and assigning LoD to the other points as points belonging to that LoD.

[0201] For example, the lowest LoD is set to "0", the next higher LoD is set to "1" (corresponding to a layer with a division count of 2 in the example in Figure 13), and so on, increasing the LoD by 1 each time up to the division count of 0.

[0202] For example, when the LoD value is an even number, it may be determined that the node with the smallest Morton Code or Partition-based Code value among the nodes belonging to the same parent node is sent to the higher LoD, and when the LoD value is an odd number, the node with the largest Morton Code or Partition-based Code value among the nodes belonging to the same parent node is sent to the higher LoD. Alternatively, the above-mentioned processing when the LoD value is an odd number and the processing when the LoD value is an even number may be interchanged.

[0203] Specific processing will be described below for the cases where Morton Code and Partition-based Code are used.

[0204] First, an example of the case where, in step S1201, LoD is calculated for position information data of each point sorted by the above-mentioned calculation of Morton Code and the method described with reference to FIG. 14 will be described.

[0205] First, the LoD calculation unit 2090 starts processing from the lowest LoD (LoD "0"). For example, in the example of FIG. 13, the size of the parent node of the node (size (x, y) = (1, 1)) corresponding to the lowest LoD is (x, y) = (1, 2). Using this, only the ones digit and above for the bits corresponding to x in the Morton Code and the twos digit and above for the bits corresponding to y are focused on.

[0206] Specifically, attention is focused only on the area enclosed in a square in the "parent node size = (1, 2)" column in FIG.

[0207] Secondly, the LoD calculation unit 2090 groups nodes whose rectangular parts have the same pattern. For example, nodes a and b, nodes c and d, and nodes e and f each have the same pattern. This grouping can also be achieved by loop processing.

[0208] For example, first, the LoD calculation unit 2090 extracts and stores the pattern of the part enclosed by the above-mentioned square from the Morton Code of the top node a. The LoD calculation unit 2090 also sets this node as the starting point.

[0209] Secondly, the LoD calculation unit 2090 processes node b and node c in order, and similarly extracts the pattern of the part enclosed by the square, and compares it with the pattern of node a previously saved. At this time, if the pattern differs from the saved pattern, the LoD calculation unit 2090 sets the node checked most recently (the node with the largest Morton Code among the nodes having the same pattern as the start point) as the end point.

[0210] Third, the LoD calculation unit 2090 sends one node from among the nodes between the start point and the end point to a higher LoD, and sets the other nodes to that LoD.

[0211] Fourth, the LoD calculation unit 2090 sets the node having a pattern different from the above-mentioned stored pattern as a new starting point, and repeats the above-mentioned process for all nodes.

[0212] Such grouping corresponds to a group of child nodes having the same parent node. Therefore, the LoD calculation unit 2090 checks all bit values ​​of the Morton Code in each group and sends the node with the smallest or largest value to the higher LoD. The example in Fig. 15 shows a case where the node with the smaller Morton Code is sent to the higher LoD.

[0213] "Parent node size = (2, 4)" in Fig. 15 corresponds to processing of LoD "1". The nodes in the "parent node size = (2, 4)" column are points sent to the LoD in question from the lowest LoD mentioned above. The LoD in question corresponds to the division count 2 in Fig. 13. The size of the parent node of the node corresponding to the LoD in question is (x, y) = (2, 4).

[0214] Therefore, as in the above case, attention is paid to only the 2's digit and above for the bits corresponding to x in the Morton Code, and only the 4's digit and above for the bits corresponding to y.

[0215] Fifth, the LoD calculation unit 2090 performs grouping in the same manner as described above, and sends one node from the same group to a higher LoD. The example in Fig. 15 is an example in which the node with the largest Morton Code in the same group is sent to the higher LoD.

[0216] By repeating the above process up to the LoD corresponding to the division number 0 in FIG. 13, the LoD of all points can be determined.

[0217] That is, the LoD calculation unit 2090 calculates the LoD of each point by referring to the bits of the Morton Code corresponding to the digits equal to or greater than the node size.

[0218] In the above, the case where the variable skipOctreeLayers has a value of "0" has been described as an example. For example, when the variable skipOctreeLayers has a value of "1", the processing of the lower layer is skipped, and processing starts from the column "parent node size = (2, 4)" in Fig. 15. Similarly, when the variable skipOctreeLayers has a value of "N", processing of the lower N layers is skipped.

[0219] Next, we will explain the case where partition-based codes are used. When partition-based codes are used, the LoD is basically determined by grouping the nodes as described above, sending one node per group to a higher LoD, and treating the other nodes as belonging to that LoD.

[0220] In the case of Partition-based Code, as explained using FIG. 13, in two dimensions, each pair of two bits of "yx" corresponds to each partition. Therefore, in the processing of the lowest LoD, the lowest two bits are ignored and grouping is performed using the pattern of the highest four bits, making it possible to group child nodes belonging to the same parent node. Similarly, in the second lowest LoD, grouping can be performed using the pattern of the highest two bits of the Partition-based Code. Subsequent processing is the same as in the case of Morton Code described above.

[0221] After the LoD of each node is calculated as described above, the process proceeds to step S1203.

[0222] In step S1203, the LoD calculation unit 2090 performs k-nearest neighbor search for nodes that belong to a higher LoD than the LoD of the node. In the k-nearest neighbor search, it is necessary to calculate the distance between each node. At this time, when scalable lifting is applied and the value of skipOctreeLayers is "1" or more, the node size is larger than (x, y, z) = (1, 1, 1), and the exact position of the point cannot be known.

[0223] Therefore, the LoD calculation unit 2090 calculates the distance by using the coordinate values ​​quantized using the node size at each LoD as the coordinate values ​​of the node. Specifically, as described below, the LoD calculation unit 2090 may quantize the coordinates so that they are at the center of each node.

[0224] ShiftX=Log2(x-direction size of the node) x=(x coordinate of the node >>ShiftX)< <shiftx + 当該ノードのx方向のサイズ 2 shifty="Log2(当該ノードのy方向のサイズ)" y="(当該ノードのy座標">>ShiftY)< <shifty + 当該ノードのy方向のサイズ 2 shiftz="Log2(当該ノードのz方向のサイズ)" z="(当該ノードのz座標">>ShiftZ) << ShiftZ + half of the size of the node in the z direction For example, in the example of FIG. 13, when the value of skipOctreeLayers is "1", the size of the node in the x direction is "1", and the size of the node in the y direction is "2". Therefore, taking the logarithm with base 2 respectively, ShiftX is "0" and ShiftY is "1".

[0225] As a result of the above quantization, when the coordinates of two nodes become exactly the same, for example, the distance (L2 norm) between the two may be defined as the square of the length of the shorter side of the node size.

[0226] That is, the LoD calculation unit 2090 performs k-nearest neighbor search between the nodes belonging to the LoD and the nodes belonging to a higher level than the LoD for each LoD. When "scalable lifting" is applied to the slice in the above k-nearest neighbor search, the central coordinates of the node in the LoD are calculated with reference to the node sizes of the x, y, and z axes respectively, and the central coordinates are regarded as the coordinates of the node.

[0227] The LoD calculation unit 2090 performs k-nearest neighbor search using the coordinates of the nodes obtained as described above. For the specific processing of k-nearest neighbor search, the methods described in Non-Patent Document 1 and Non-Patent Document 2 can be applied.

[0228] After performing k-nearest neighbor search for all nodes, this process proceeds to step S1204 and ends.

[0229] In addition, the above-described point cloud encoding device 100 and point cloud decoding device 200 may be realized by a program that causes a computer to execute each function (each process).

[0230] In each of the above embodiments, the present invention has been described using the point cloud encoding device 100 and the point cloud decoding device 200 as examples, but the present invention is not limited to such examples and can be similarly applied to a point cloud encoding / decoding system having the functions of the point cloud encoding device 100 and the point cloud decoding device 200. [Explanation of symbols]

[0231] 10...Point cloud processing system 100...Point cloud encoding device 200...Point cloud decoding device 2010…Geometric Information Decoding Department 2020…Tree synthesis section 2030…Approximate surface synthesis part 2040...Geometric information reconstruction unit 2050…Inverse coordinate conversion section 2060…Attribute information decoding unit 2070...Inverse quantization section 2080…RAHT Division 2090…LoD calculation section 2100…Reverse lifting section 2110…Color inversion unit< / shifty> < / shiftx>

Claims

1. A point cloud decoding device, comprising: A geometry information decoding unit configured to decode a flag that controls whether to apply “Implicit QtBt”; an attribute information decoding unit configured to decode a flag that controls whether to apply "scalable lifting"; A point cloud decoding device characterized in that, when the "Implicit QtBt" is applied, the "Scalable Lifting" is restricted not to be applied.

2. Decoding a flag that controls whether to apply "Implicit QtBt"; and decoding a flag that controls whether to apply "scalable lifting"; A point cloud decoding method characterized in that, when the "Implicit QtBt" is applied, the "Scalable Lifting" is restricted not to be applied.

3. A program for causing a computer to function as a point group decoding device, The point group decoding device comprises: A geometry information decoding unit configured to decode a flag that controls whether to apply “Implicit QtBt”; an attribute information decoding unit configured to decode a flag that controls whether to apply "scalable lifting"; A program characterized in that, when the "Implicit QtBt" is applied, the "Scalable Lifting" is restricted not to be applied.

Images