Point group decoding device, point group decoding method and program

Abstract

To uniquely associate decoded position information with attribute information by uniquely determining an order for performing decoding processing of attribute information even when a sorting process is not performed when decoding attribute information.SOLUTION: A point cloud decoding device 200 according to the present invention includes a tree synthesis unit 2020 configured to generate a flag indicating whether the planar mode can be applied for each axial direction of a node, and the tree synthesis unit 2020 is configured to set the value of the flag indicating whether the planar mode can be applied corresponding to the axial direction in which no division occurs, to a value indicating that the planar mode cannot be applied, when a node is divided into a quadtree or binary tree.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a point group decoding device, a point group decoding method, and a program. [Background technology]

[0002] Non-Patent Document 1 discloses a technique in which position information of each point of point cloud data is decoded, and then each point of the point cloud data is sorted based on the position information prior to decoding attribute information of each point of the point cloud data. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Text of ISO / IEC 23090-9 DIS Geometry-based PCC, ISO / IEC / JTC1 / SC29 / WG11 N19088 [Non-Patent Document 2] [G-PCC][New Proposal] Predictive Geometry Coding, ISO / IEC JTC1 / SC29 / WG11 m51012 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, when sorting each point of the point cloud data based on its position information, there was a problem that decoding of the attribute information could not start until decoding of the position information was completed for all points of the point cloud.

[0005] Furthermore, in the conventional technology, when the sorting process is omitted, there is a problem that the processing order of the point groups in the attribute information decoding process is not defined.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a point cloud decoding device, a point cloud decoding method, and a program that can uniquely determine the order in which the attribute information decoding process is performed, even if a sorting process is not performed when decoding the attribute information, thereby uniquely associating the decoded position information with the attribute information. [Means for solving the problem]

[0007] A first feature of the present invention is summarized as a point cloud decoding device including a tree synthesis unit configured to decode position information for each point of point cloud data to be decoded, assign an index to each point of the point cloud data in ascending order of decoding of the position information, and store the index in a one-dimensional array.

[0008] A second feature of the present invention is a point cloud decoding device comprising a tree synthesis unit configured to generate Mask_planar corresponding to eight child nodes of a node, wherein when the node is quadtree divided or binary tree divided, the tree synthesis unit is configured to set the value of Mask_planar corresponding to a child node belonging to a plane closer to the origin in the undivided axial direction to "0", and to set the value of Mask_planar corresponding to a child node belonging to a plane farther from the origin in the undivided axial direction to "1".

[0009] A third feature of the present invention is summarized as a point cloud decoding device including a tree synthesis unit configured to generate a flag indicating whether or not a planar mode can be applied for each axial direction of a node, and the tree synthesis unit is configured to set, when the node is divided into a quadtree or a binary tree, a value of the flag indicating whether or not the planar mode can be applied corresponding to an axial direction in which no division occurs, to a value indicating that the planar mode cannot be applied.

[0010] A fourth feature of the present invention is summarized as a point cloud decoding method comprising the steps of: decoding position information for each point of point cloud data to be decoded; assigning an index to each point of the point cloud data in ascending order of decoding of the position information; and storing the index in a one-dimensional array.

[0011] A fifth feature of the present invention is a program for use in a point cloud decoding device, which causes a computer to execute the steps of decoding position information for each point of point cloud data to be decoded, assigning an index to each point of the point cloud data in ascending order of decoding of the position information, and storing the index in a one-dimensional array. Effect of the Invention

[0012] According to the present invention, it is possible to provide a point cloud decoding device, a point cloud decoding method, and a program that can uniquely determine the order in which the attribute information decoding process is performed, even if a sorting process is not performed when decoding the attribute information, thereby uniquely associating the decoded position information with the attribute information. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a point cloud processing system 10 according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of functional blocks of a point group decoding device 200 according to an embodiment. [Diagram 3] 1 shows an example of the configuration of encoded data (bit stream) received by a geometric information decoding unit 2010 of a point cloud decoding device 200 according to an embodiment. [Figure 4] 1 is an example of a syntax configuration of GPS2011 according to an embodiment. [Diagram 5] 13 is a flowchart showing an example of processing by a tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. [Figure 6] 7 is a flowchart showing an example of the node information decoding process in step S506 of FIG. 5. [Figure 7]7 is a flowchart showing an example of the node information decoding process in step S506 of FIG. 5. [Figure 8] FIG. 7 is a diagram showing an example of a method for setting Mask_planar in step S601 of FIG. [Figure 9] 13 is an example of the configuration of encoded data (bit stream) received by an attribute information decoding unit 2060 of the point group decoding device 200 according to an embodiment. [Figure 10] 1 is an example of a syntax configuration of APS2061 according to an embodiment. [Figure 11] 11 is a flowchart illustrating an example of a processing procedure of a LoD calculation unit 2090 of the point cloud decoding device 200 according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0015] (First embodiment) A point cloud processing system 10 according to a first embodiment of the present invention will be described below with reference to Fig. 1 to Fig. 11. Fig. 1 is a diagram showing a point cloud processing system 10 according to the embodiment.

[0016] As shown in FIG. 1, a point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0017] The point cloud encoding device 100 is configured to generate encoded data (bit stream) by encoding an input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bit stream.

[0018] The input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud, such as color information and reflectance of each point.

[0019] Here, such a bit stream may be transmitted from the point group encoding device 100 to the point group decoding device 200 via a transmission path. Also, the bit stream may be stored in a storage medium and then provided from the point group encoding device 100 to the point group decoding device 200.

[0020] (Point Cloud Decoding Device 200) Hereinafter, the point group decoding device 200 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the point group decoding device 200 according to this embodiment.

[0021] As shown in FIG. 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, an LoD calculation unit 2090, an inverse lifting unit 2100, and an inverse color transformation unit 2110.

[0022] The geometric information decoding unit 2010 is configured to receive as input a bit stream relating to geometric information (geometric information bit stream) out of the bit streams output from the point group encoding device 100, and to decode the syntax.

[0023] The decoding process is, for example, a context adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0024] The tree synthesis unit 2020 is configured to receive as input the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating which node in the tree described below the point group exists in, and generate tree information indicating in which area in the space to be decoded the point exists.

[0025] This process divides the space to be decoded into rectangles, determines whether a point exists in each rectangle by referring to the occupancy code, divides the rectangle in which the point exists into multiple rectangles, and then generates tree information by recursively repeating the process of referring to the occupancy code.

[0026] In this embodiment, a method called "Octree" that always treats the above-mentioned rectangular parallelepiped as a cube and performs octree division recursively, and a method called "QtBt" that performs quadtree division and binary tree division in addition to octree division can be used. Whether or not to use "QtBt" is transmitted from the point cloud encoding device 100 as control data.

[0027] Alternatively, when the control data specifies the use of predicitive coding, which will be described later, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree structure determined in the point cloud encoding device 100.

[0028] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree information synthesis unit 2020.

[0029] Approximate surface information is used when, for example, decoding three-dimensional point cloud data of an object, in cases where the point cloud is densely distributed on the object surface, to approximate the area in which the point cloud exists using a small plane, rather than decoding each individual point cloud.

[0030] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information, for example, by a method called "Trisoup". As a specific process of "Trisoup", for example, the method described in Non-Patent Document 1 can be used. In addition, when decoding a sparse point group acquired by Lidar or the like, this process can be omitted.

[0031] The geometric information reconstruction unit 2040 is configured to reconstruct geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point cloud data to be decoded, based on the tree information generated by the tree information synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0032] The inverse coordinate transformation unit 2050 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform the information from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

[0033] The attribute information decoding unit 2060 is configured to receive as input a bit stream relating to attribute information (attribute information bit stream) out of the bit streams output from the point group encoding device 100, and to decode the syntax.

[0034] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0035] Moreover, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

[0036] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and a quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, to generate inverse quantized residual information.

[0037] The dequantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090 according to the features of the point group to be decoded. Control data decoded by the attribute information decoding unit 2060 specifies which unit the dequantized residual information is output to.

[0038] The RAHT unit 2080 is configured to receive the inverse quantized residual information generated by the inverse quantized residual information and the geometric information generated by the geometric information reconstruction unit 2040, and to decode the attribute information of each point using a type of Haar transform (inverse Haar transform in the decoding process) called RAHT (Region Adaptive Hierarchical Transform). As a specific process of RAHT, for example, the method described in Non-Patent Document 1 can be used.

[0039] The LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 as an input and generate a Level of Detail (LoD).

[0040] LoD is information for defining a reference relationship (a referencing point and a referenced point) to realize predictive coding, such as predicting attribute information of a certain point from attribute information of another point and encoding or decoding the prediction residual.

[0041] In other words, LoD is information that defines a hierarchical structure in which each point contained in geometric information is classified into multiple levels, and attributes of points belonging to lower levels are encoded or decoded using attribute information of points belonging to higher levels.

[0042] As a specific method for determining the LoD, for example, the method described in Non-Patent Document 1 may be used.

[0043] The inverse lifting unit 2100 is configured to decode attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse quantized residual information generated by the inverse quantized residual information. As a specific process of the inverse lifting, for example, the method described in Non-Patent Document 1 can be used.

[0044] The inverse color conversion unit 2110 is configured to perform inverse color conversion processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information to be decoded is color information and color conversion has been performed on the point group encoding device 100 side. Whether or not to perform such inverse color conversion processing is determined by the control data decoded by the attribute information decoding unit 2060.

[0045] The point group decoding device 200 is configured to decode and output the attribute information of each point in the point group through the above processing.

[0046] (Geometric Information Decoding Part 2010) The control data decoded by the geometric information decoding unit 2010 will be described below with reference to FIGS.

[0047] FIG. 3 shows an example of the structure of the coded data (bit stream) received by the geometric information decoding unit 2010. In FIG.

[0048] First, the bit stream may include a GPS 2011. The GPS 2011 is also called a geometry parameter set, and is a set of control data related to decoding of geometric information. A specific example will be described later. Each GPS 2011 includes at least GPS ID information for identifying each GPS 2011 when there are multiple GPS 2011s.

[0049] Secondly, the bit stream may include GSH2012A / 2012B. GSH2012A / 2012B is also called a geometry slice header or geometry data unit header, and is a set of control data corresponding to a slice described later. In the following description, the term "slice" is used, but slice can also be read as data unit. A specific example will be described later. GSH2012A / 2012B includes at least GPS id information for specifying GPS2011 corresponding to each GSH2012A / 2012B.

[0050] Thirdly, the bit stream may include slice data 2013A / 2013B following the GSH 2012A / 2012B. The slice data 2013A / 2013B includes data that encodes geometric information. An example of the slice data 2013A / 2013B is an occupancy code, which will be described later.

[0051] As described above, the bit stream is configured such that each slice data 2013A / 2013B corresponds to one GSH 2012A / 2012B and one GPS 2011.

[0052] As described above, since the GPS ID information is used to specify which GPS 2011 to refer to in the GSH 2012A / 2012B, a common GPS 2011 can be used for a plurality of slice data 2013A / 2013B.

[0053] In other words, it is not necessary to transmit the GPS 2011 for each slice. For example, as shown in Fig. 3, the bit stream may be configured such that the GPS 2011 is not coded immediately before the GSH 2012B and the slice data 2013B.

[0054] The configuration in Fig. 3 is merely an example. As long as the GSH 2012A / 2012B and the GPS 2011 correspond to each slice data 2013A / 2013B, elements other than those described above may be added as components of the bit stream. For example, the bit stream may include a sequence parameter set (SPS). Similarly, the bit stream may be shaped into a different configuration from that in Fig. 3 when transmitted. Furthermore, the bit stream may be combined with a bit stream decoded by an attribute information decoding unit 2060 (described later) and transmitted as a single bit stream.

[0055] FIG. 4 is an example of the syntax configuration of GPS2011.

[0056] Note that the syntax names described below are merely examples. If the syntax functions described below are similar, the syntax names may be different.

[0057] The GPS 2011 may include GPS ID information (gps_geom_parameter_set_id) for identifying each GPS 2011.

[0058] The Descriptor column in Fig. 4 indicates how each syntax is coded. ue(v) indicates an unsigned zeroth-order exponential Golomb code, and u(1) indicates a 1-bit flag.

[0059] The GPS 2011 may include a flag (gps_predictive_mode_enabled_flag) for controlling whether or not the tree synthesis unit 2020 uses predicitive coding.

[0060] For example, when the value of gps_predictive_mode_enabled_flag is "1", it may be defined that predicitive coding is used, and when the value of gps_predictive_mode_enabled_flag is "0", it may be defined that predicitive coding is not used.

[0061] The GPS 2011 may include a flag (geometry_planar_mode_flag) that controls whether the tree synthesis unit 2020 uses the planar mode.

[0062] For example, when the value of geometry_planar_mode_flag is "1", it may be defined that planar mode is used, and when the value of geometry_planar_mode_flag is "0", it may be defined that planar mode is not used.

[0063] The GPS 2011 may include a flag (geom_tree_coded_axis_list_present_flag) for controlling whether or not the tree synthesis unit 2020 performs “QtBt”.

[0064] For example, if the value of geom_tree_coded_axis_list_present_flag is "1", it may be defined that "QtBt" is performed, and if the value of geom_tree_coded_axis_list_present_flag is "0", it may be defined that only "Octree" is performed.

[0065] Alternatively, if the value of geom_tree_coded_axis_list_present_flag is "1", it can be defined that node shapes other than cubes are also allowed (i.e., "QtBt" is also used), and if the value of geom_tree_coded_axis_list_present_flag is "0", it can be defined that only cubes are allowed as node shapes (i.e., only "Octree" is used).

[0066] Any definition other than the above may be used as long as it controls whether "QtBt" is available.

[0067] In addition, geometry_planar_mode_flag and geom_tree_coded_axis_list_present_flag may be defined to be decoded only when predicitive coding is not used.

[0068] (Tree Synthesis Department 2020) An example of the processing performed by the tree merging unit 2020 will be described with reference to Figures 5 to 8. Figure 5 is a flowchart showing an example of the processing performed by the tree merging unit 2020.

[0069] As shown in FIG. 5, in step S501, the tree synthesis unit 2020 determines whether or not to apply predicitive coding.

[0070] For example, the tree synthesis unit 2020 can refer to the value of gps_predictive_mode_enabled_flag to determine whether to apply predicitive coding.

[0071] If the tree synthesis unit 2020 determines that predicitive coding is to be applied, the process proceeds to step S502, and if it determines that predicitive coding is not to be applied, the process proceeds to step S504.

[0072] In step S502, the tree synthesis unit 2020 determines whether the decoding of the position information of all point cloud data included in the corresponding slice has been completed.

[0073] The processing of step S502 can, for example, transmit information indicating the number of point cloud data contained in the slice to the GSH, and determine whether processing of all points has been completed by comparing the number of point cloud data with the number of data already processed.

[0074] If the tree merging unit 2020 has completed decoding of the position information of all point cloud data (position information of all points in the point cloud), the tree merging unit 2020 proceeds to processing in step S507 and ends the processing. On the other hand, if the tree merging unit 2020 has not completed decoding of the position information of all point cloud data (position information of all points in the point cloud), the tree merging unit 2020 proceeds to processing in step S503.

[0075] In step S503, the tree synthesis unit 2020 decodes position information of one point of the point cloud data (position information of one point in the point cloud) by predicitive coding.

[0076] Predicitive coding is a method for decoding the position information of point cloud data by decoding the difference between position information predicted from decoded point cloud data and the position information of the point cloud data, and adding the two together. A specific method can be realized, for example, by the method described in Non-Patent Document 2.

[0077] After the tree synthesis unit 2020 decodes the position information of one point of the point cloud data in step S503, the process returns to step S502.

[0078] When it is determined that predicitive coding is to be applied, the tree synthesis unit 2020 can decode the position information of all the point cloud data by repeatedly executing steps S502 and S503.

[0079] Here, the tree synthesis unit 2020 may assign an index to each point of the point group data whose position information is decoded in ascending order of the order decoded in step S503. Furthermore, the tree synthesis unit 2020 may store such indexes in order in one-dimensional array data Order[i].

[0080] As described above, the tree synthesis unit 2020 may be configured to include predicitive coding as a means for decoding position information, and to assign an index to each point of the point cloud data based on the decoding order in the predicitive coding, and to store the index in a one-dimensional array.

[0081] With this configuration, even if the sorting process is not performed in the LoD calculation unit 2090 described later, the order in which the attribute information is decoded is uniquely determined, so that the decoded position information and the attribute information are uniquely associated with each other.

[0082] Next, a process in which predictive coding is not applied will be described.

[0083] In step S504, the tree synthesis unit 2020 checks whether or not processing of all depths has been completed for the tree structure when "Octree" or "QtBt" is applied.

[0084] The depth value of the slice can be transmitted, for example, by GSH. The tree synthesis unit 2020 can determine whether the processing for all depths is completed by comparing the depth value of the slice with the depth values ​​that have already been processed.

[0085] If the tree merging unit 2020 has completed the processing for all depths, the tree merging unit 2020 proceeds to the processing of step S507 and ends the processing. On the other hand, if the tree merging unit 2020 has not completed the processing for all depths, the tree merging unit 2020 proceeds to the processing of step S505.

[0086] In step S505, the tree synthesis unit 2020 determines whether or not the processing has been completed for all nodes within the relevant Depth.

[0087] The number of nodes included in the Depth can be calculated from the result of the node information decoding process (step S506) at the Depth processed immediately before. The number of nodes included in the Depth processed first is 1.

[0088] If the tree merging unit 2020 has completed processing of all nodes included in the Depth, the tree merging unit 2020 returns to the process of step S504. On the other hand, if the tree merging unit 2020 has not completed processing of all nodes, the tree merging unit 2020 proceeds to the process of step S506.

[0089] In step S506, if "Octree" is used, the tree synthesis unit 2020 divides the node into 8 child nodes, and if "QtBt" is used, the tree synthesis unit 2020 divides the node into 8, 4, or 2 child nodes, and decodes an occupancy map that indicates whether each child node contains point cloud data based on information such as the occupancy code.

[0090] For example, when the value of the occupancy map is "1", it can be defined that point cloud data exists in the corresponding child node. Also, when the value of the occupancy map is "0", it can be defined that point cloud data does not exist in the corresponding child node or that the child node itself does not exist.

[0091] Therefore, the tree synthesis unit 2020 can know the number of nodes in the next depth to be processed by counting the number of child nodes in the same depth whose occupancy map value is "1." A specific example of the process in step S506 will be described later.

[0092] When the tree synthesis unit 2020 completes the generation of the occupancy map for one node, the process returns to step S505.

[0093] As described above, when predicitive coding is not applied, the tree synthesis unit 2020 can decode the position information of the point cloud data by repeating the processing of steps S504 to S506 and recursively dividing the node until the node size becomes 1 x 1 x 1.

[0094] Here, for each point of the point group data whose position information has been decoded, an index may be assigned in ascending order in the order in which it was decoded in step S506, and the points may be stored in one-dimensional array data Order[i] in the order of index.

[0095] As described above, the tree synthesis unit 2020 may decode position information for each point of the point cloud data to be decoded, and assign an index to each point in ascending order of the decoded position information. Furthermore, the tree synthesis unit 2020 may be configured to store such indexes in a one-dimensional array.

[0096] With this configuration, even if the sorting process is not performed in the LoD calculation unit 2090 described later, the order in which the attribute information is decoded is uniquely determined, so that the decoded position information and the attribute information are uniquely associated with each other.

[0097] FIG. 6 is a flowchart showing an example of the node information decoding process in step S506.

[0098] In step S601, the tree synthesis unit 2020 determines whether or not the planar mode is applicable for each axis direction of the node (for example, each of the x-axis, y-axis, and z-axis).

[0099] If the planar mode is applicable, the tree merging unit 2020 proceeds to the process of step S602. On the other hand, if the planar mode is not applicable, the tree merging unit 2020 proceeds to the process of step S603.

[0100] The tree synthesis unit 2020 can determine whether the planar mode is applicable by, for example, first deriving an eligible_planar_flag that indicates whether the planar mode is applicable for each axial direction, and referring to the value of the eligible_planar_flag.

[0101] For example, when the value of eligible_planar_flag is "1", it may be defined that the planar mode can be applied in the corresponding axis direction of the node, and when the value of eligible_planar_flag is "0", it may be defined that the planar mode cannot be applied in the corresponding axis direction of the node.

[0102] Here, if the axis direction of the node is not divided by applying "QtBt", the value of the corresponding eligible_planar_flag may be set to "0", that is, the planar mode may not be applicable.

[0103] In other words, a flag indicating whether or not the planar mode can be applied may be generated for each axial direction of the node, and when the node is divided into quadtrees or binary trees, the value of the flag indicating whether or not the planar mode can be applied corresponding to the axial direction in which no division occurs may be set to a value indicating that the planar mode cannot be applied.

[0104] Here, whether or not the axis direction of the node is to be divided can be determined using various methods. For example, as in Non-Patent Document 1, it may be implicitly determined based on a predetermined rule and node shape, or it may be explicitly determined on the point cloud encoding device 100 side and transmitted via a bit stream.

[0105] For example, information on which axis direction to divide at each depth may be transmitted to the GSH as a syntax of 3 bits (each bit corresponding to each axis direction). For example, it can be defined that an axis with a bit value of "1" is to be divided, and an axis with a bit value of "0" is not to be divided.

[0106] Here, for example, if the bit corresponding to the axis direction at the Depth to which the node belongs indicates "0 (= "do not split")", the value of the corresponding eligible_planar_flag may be set to "0", i.e., planar mode is not applicable.

[0107] In step S602, the tree synthesis unit 2020 decodes syntax related to the planar mode. For example, the tree synthesis unit 2020 decodes is_planar_flag and plane_position described in Non-Patent Document 1. After decoding the syntax, the tree synthesis unit 2020 proceeds to the process of step S603.

[0108] In step S603, the tree synthesis unit 2020 sets Mask_planar.

[0109] Mask_planar is a bit mask consisting of 8 bits. Each bit corresponds to a child node when the node is divided into two along each of the three axes (e.g., x-axis, y-axis, z-axis). For example, when a bit has a value of "1", it can be defined that the corresponding child node does not exist or that the corresponding child node does not have point cloud data. Similarly, when a bit has a value of "0", it can be defined that the corresponding child node may have point cloud data.

[0110] Here, the tree synthesis unit 2020 may set Mask_planar so that for an axis direction in which no splitting occurred due to "QtBt", when viewed along that axis, the bit corresponding to the child node belonging to the plane (Lower plane) whose coordinates are closer to the origin (0) is "0", and the bit corresponding to the child node belonging to the plane (Upper plane) whose coordinates are farther from the origin is "1".

[0111] In other words, the tree synthesis unit 2020 may be configured to generate Mask_planar corresponding to the eight child nodes of each node, and when the node is divided into quadtrees or binary trees, to set the value of Mask_planar corresponding to the child node belonging to the plane that is not divided and closer to the origin in the axial direction to "0", and to set the value of Mask_planar corresponding to the child node belonging to the plane that is not divided and farther from the origin in the axial direction to "1".

[0112] Such a specific example is shown in FIG.

[0113] For example, as shown in Fig. 8(a), when the node is Qt (quadtree partitioning) so as not to be partitioned in the z-axis direction, the bit of Mask_planar corresponding to the Lower plane in the z-axis direction becomes "0" and the bit corresponding to the Upper plane in the z-axis direction becomes "1" as shown in Fig. 8(a). Note that the labels a to h described as positions in Fig. 8(a) correspond to the positions of each child node described in Fig. 8(c).

[0114] Similarly, when Bt (binary tree partitioning) is performed so that neither the x-axis nor the y-axis directions are partitioned, as in Figure 8(b), only the bits corresponding to the child nodes belonging to the Lower plane in both the x-axis and y-axis directions will be "0", and the other bits will be "1".

[0115] As described above, after the tree merging unit 2020 sets Mask_planar, the process proceeds to step S604.

[0116] In step S604, the tree synthesis unit 2020 decodes the occupancy map. Here, the above-mentioned Mask_planar and the occupancy code included in the bitstream transmitted from the point group encoding device 100 are used to decode the occupancy map.

[0117] The occupancy map is composed of 8 bits, similar to the above-mentioned Mask_planar. Each bit corresponds to each child node when the node is divided into two along each of the three axes (for example, the x-axis, y-axis, and z-axis), similar to the Mask_planar.

[0118] Here, for a child node in which the value of Mask_planar is "1", the tree synthesis unit 2020 decodes the corresponding bit of the occupancy map as "0 (point cloud data does not exist in the child node)".

[0119] On the other hand, for a child node whose Mask_planar value is "0", the tree synthesis unit 2020 decodes the Occupancy code, and if the value of the Occupancy code is "1 (point cloud data exists in the child node)", it sets the corresponding bit of the Occupancy map to "1", and if the value of the Occupancy code is "0 (point cloud data does not exist in the child node)", it sets the corresponding bit of the Occupancy map to "0".

[0120] In other words, the tree synthesis unit 2020 may be configured to decode an Occupancy map indicating whether or not point cloud data exists in each child node of the node, and for a child node having a value of "1" for the above-mentioned Mask_planar, set the corresponding Occupancy map value to "0", and for a child node having a value of "0" for the above-mentioned Mask_planar, set the value of the Occupancy map by referring to the value of the Occupancy code included in the bit stream.

[0121] With the above configuration, control during "QtBt" can be achieved using Mask_planar provided for planar mode, simplifying the specifications and the design of devices based on them.

[0122] The above-described value of Mask_planar is merely an example, and the meanings of the bits "0" and "1" described above may be interchanged.

[0123] As described above, the tree synthesis unit 2020 can decode the 8-bit occupancy map corresponding to each node. After decoding the occupancy map, the tree synthesis unit 2020 proceeds to the process of step S605 and ends the process.

[0124] The order of the above steps can be changed as shown in Fig. 7. Specifically, the tree synthesis unit 2020 can first perform Mask_planar setting and Occupancy map decoding, and then determine whether or not the planar mode can be used and decode the syntax related to the planar mode.

[0125] In this case, the determination of whether the planar mode can be used and the decoding process of the syntax related to the planar mode are executed not for the node in question but for each child node of the node in question. Therefore, step S701 is added, and the tree synthesis unit 2020 performs a loop process until the process is completed for all child nodes decoded as having point cloud data in the occupancy map.

[0126] (Attribute information decoding unit 2060) The control data decoded by the attribute information decoding unit 2060 will be described below with reference to FIGS.

[0127] FIG. 9 shows an example of the structure of the coded data (bit stream) received by the geometric information decoding unit 2060. In FIG.

[0128] First, the bit stream may include APS2061. APS2061 is also called an attribute parameter set, and is a set of control data related to the decoding of attribute information. A specific example will be described later. Each APS2061 includes at least APS id information for identifying each APS2061 when multiple APS2061 exist.

[0129] Secondly, the bitstream may include ASH2062A / 2062B. ASH2062A / 2062B is also called an attribute slice header and is a set of control data corresponding to slices described later. A specific example will be described later. ASH2062A / 2062B includes at least APS id information for specifying APS2061 corresponding to each ASH2062A / 2062B.

[0130] Thirdly, the bit stream may include slice data 2063A / 2063B following ASH 2062A / 2062B. The slice data 2063A / 2063B includes data in which attribute information is encoded.

[0131] As described above, the bit stream is configured such that each slice data 2063A / 2063B corresponds to one ASH 2062A / 2062B and one APS 2061.

[0132] As described above, in ASH2062A / 2062B, which APS2061 to refer to is specified by APS id information, so that a common APS2061 can be used for a plurality of slice data 2063A / 2063B.

[0133] In other words, it is not necessary to transmit the APS 2061 for each slice. For example, as shown in Fig. 9, the bit stream may be configured such that the APS 2061 is not coded immediately before the ASH 2062B and slice data 2063B.

[0134] Note that the configuration in Fig. 9 is merely an example. As long as the ASH2062A / 2062B and the APS2061 correspond to each slice data 2063A / 2063B, elements other than those described above may be added as components of the bit stream. For example, the bit stream may include a sequence parameter set (SPS).

[0135] Similarly, when transmitted, the data may be shaped into a configuration different from that shown in FIG. 9. Furthermore, the data may be combined with the bit stream decoded by the geometric information decoding unit 2010 and transmitted as a single bit stream. For example, the slice data 2013A and 2063A, and the slice data 2013B and 2063B may each be treated as a single slice data, and GSH2012A and ASH2062A, and GSH2012B and ASH2062B may each be arranged immediately before each slice. In this case, GPS2011 and APS2061 may be arranged prior to each GSH and ASH.

[0136] FIG. 10 is an example of the syntax configuration of APS2061.

[0137] The APS 2061 may include APS id information (aps_attr_parameter_set_id) for identifying each APS 2061.

[0138] The APS 2061 may include information (attr_coding_type) indicating a decoding method of the attribute information. For example, it may be specified that when the value of attr_coding_type is "0", the inverse lifting unit 2100 performs variable weighted lifting prediction, when the value of attr_coding_type is "1", the RAHT unit 2080 performs RAHT, and when the value of attr_coding_type is "2", the inverse lifting unit 2100 performs lifting prediction with a fixed weight.

[0139] The APS 2061 may include a flag (lifting_scalability_enabled_flag) indicating whether to apply scalable lifting when the value of attr_coding_type is '2', that is, when the inverse lifting unit 2100 performs lifting prediction with fixed weights.

[0140] When the value of lifting_scalability_enabled_flag is "0", it may be specified that scalable lifting is not applied, and when the value of lifting_scalability_enabled_flag is "1", it may be specified that scalable lifting is applied.

[0141] Furthermore, when the value of lifting_scalability_enabled_flag is "1", that is, when scalable lifting is applied, it may be specified that skipOctreeLayers is acquired as an external parameter at run time of the image decoding device 200, as described above.

[0142] In addition, when the above-mentioned "QtBt" is applied, i.e., when the value of geom_tree_coded_axis_list_present_flag is "1", it may be specified that scalable lifting is not applied, i.e., the value of lifting_scalability_enabled_flag is always "0".

[0143] That is, the device is provided with a geometric information decoding unit 2010 that decodes a flag that controls whether or not to apply "QtBt", and an attribute information decoding unit 2060 that decodes a flag that controls whether or not to apply "scalable lifting", and may be restricted so that when "QtBt" is applied, "scalable lifting" is not applied.

[0144] In this way, by restricting the application of "QtBt" and scalable lifting exclusively, the specification can be simplified by reducing the combination of functions, making implementation easier.

[0145] For example, by restricting the application of scalable lifting to always apply "Octree", the node shape in scalable lifting processing can be performed under the assumption that octree division is always performed at each division. This simplifies the specifications compared to the case where the node shape is a rectangular prism and the division types allowed are quadtree and binary tree division in addition to octree division.

[0146] The APS 2061 may include a syntax (lifting_num_detail_levels_minus1) that specifies the number of LoD levels calculated by the LoD calculation unit 2090. Since the LoD value needs to be an integer value of 1 or more, for example, the syntax may be set to an integer value of 0 or more, and the value obtained by adding "1" to the value of the syntax may be set to the number of LoD levels.

[0147] The APS 2061 may include a flag (lifting_morton_sort_skip_enabled_flag) that controls whether or not to execute the sorting process in the LoD calculation unit 2090.

[0148] For example, it may be defined that when the value of lifting_morton_sort_skip_enabled_flag is "0", sorting is performed, and when the value of lifting_morton_sort_skip_enabled_flag is "1", sorting is not performed.

[0149] Also, for example, lifting_morton_sort_skip_enabled_flag may be defined so as to be included in APS2061 only when the value of the above-mentioned lifting_num_detail_levels_minus1 is "0", that is, when the number of LoD levels is "1".

[0150] A constraint may be defined that when the value of gps_predictive_mode_enabled_flag is “0”, i.e., when predicitive coding is not applied, the value of lifting_morton_sort_skip_enabled_flag must always be “0”.

[0151] In other words, the system may be configured to include a geometric information decoding unit 2010 that decodes a flag that controls whether or not predicitive coding is applied, and an attribute information decoding unit 2060 that decodes a flag that controls whether or not to execute a sorting process based on the decoded position information of the point cloud data to be decoded, prior to decoding of the attribute information of the point cloud data to be decoded, and to have the constraint that when the flag that controls whether or not predicitive coding is applied indicates that predicitive coding cannot be applied, the value of the flag that controls whether or not to execute the sorting process must be set to a value meaning that "sorting process is executed."

[0152] In addition, the function of "not performing sorting processing" in decoding attribute information is a function that realizes low-latency processing (reducing the delay time from when point cloud data to be encoded is input until the corresponding point cloud data is output to a bit stream) on the side of the point cloud encoding device 100. This function can be particularly effective when combined with predictive coding that sequentially decodes position information.

[0153] Therefore, by limiting the "do not perform sorting" function to only be used in combination with predicitive coding, it is possible to make design and verification easier while retaining the effectiveness of the function.

[0154] (LoD calculation unit 2090) An example of the processing contents of the LoD calculation unit 2090 will be described below with reference to FIG.

[0155] FIG. 11 is an example of a flowchart of the LoD calculation unit 2090.

[0156] In step S1101, the LoD calculation unit 2090 performs a process of rearranging the position information of each point in the point cloud output from the geometric information reconstruction unit 2040 (sorting).

[0157] Specifically, for example, the LoD calculation unit 2090 may receive one-dimensional array data Order[i] in which indexes assigned to position information of each point cloud data generated by the tree synthesis unit 2020 are stored in ascending order, and may update the one-dimensional array data Order[i] with the result of sorting based on such position information. The LoD calculation unit 2090 may sort such position information, for example, so that the Morton codes that can be generated from such position information are in ascending order.

[0158] In addition, when the flag (lifting_morton_sort_skip_enabled_flag) that controls whether or not to perform sorting processing has a value indicating that sorting processing is not to be performed, for example, in the above example, when the value of lifting_morton_sort_skip_enabled_flag is "1", the LoD calculation unit 2090 may not perform such sorting processing. When the above-mentioned sorting process is not performed, the index of each point of the point cloud data may be assigned in the decoding order in the tree synthesis unit 2020. Furthermore, the index of each point of the point cloud data may be stored in one-dimensional array data Order[i] in ascending order of the index.

[0159] If the above-mentioned sorting process is not performed and the tree synthesis unit 2020 determines to apply predicitive coding, the index of each point of the point cloud data may be assigned in the decoding order by the predicitive coding in the tree synthesis unit 2020. Furthermore, the index of each point of the point cloud data may be stored in one-dimensional array data Order[i] in ascending order of the index.

[0160] In other words, the LoD calculation unit 2090 may be configured to determine whether or not to execute a process of sorting the point cloud data based on the decoded position information prior to decoding the attribute information of the point cloud data to be decoded, and if the sorting process is not executed, to decode the attribute information based on the order of the indexes stored in the one-dimensional array.

[0161] With the above-described configuration, even if the above-described sorting process is not performed, the order in which the attribute information decoding process is performed is uniquely determined, so that the decoded position information and attribute information are uniquely associated with each other.

[0162] As described above, after the LoD calculation unit 2090 executes or omits the above-mentioned sorting process, the process proceeds to step S1102.

[0163] In step S1102, the LoD calculation unit 2090 calculates the LoD for each piece of position information. A known method can be used to calculate the LoD, and therefore detailed description thereof will be omitted.

[0164] After the LoD calculation unit 2090 calculates the LoD of each point cloud data, the process proceeds to step S1103.

[0165] In step S1103, the LoD calculation unit 2090 performs k-nearest neighbor search for nodes that belong to a higher LoD than the LoD of each node. A known method can be applied to the specific k-nearest neighbor search method, and therefore a detailed description thereof will be omitted.

[0166] After performing the k-nearest neighbor search for all nodes, the LoD calculation unit 2090 proceeds to the processing of step S1104 and ends.

[0167] Furthermore, the above-mentioned point group encoding device 100 and point group decoding device 200 may be realized as a program that causes a computer to execute each function (each process).

[0168] In each of the above embodiments, the present invention has been described using the point cloud encoding device 100 and the point cloud decoding device 200 as examples, but the present invention is not limited to such examples and can be similarly applied to a point cloud encoding / decoding system having the functions of the point cloud encoding device 100 and the point cloud decoding device 200. [Explanation of symbols]

[0169] 10...Point cloud processing system 100...Point cloud encoding device 200...Point cloud decoding device 2010…Geometric Information Decoding Department 2020…Tree synthesis section 2030…Approximate surface synthesis part 2040...Geometric information reconstruction unit 2050…Inverse coordinate conversion section 2060…Attribute information decoding unit 2070...Inverse quantization section 2080…RAHT Division 2090…LoD calculation section 2100…Reverse lifting section 2110…Color inversion unit

Claims

1. A point cloud decoding device, comprising: A tree synthesis unit configured to generate a flag indicating whether or not a planar mode is applicable for each axis direction of a node, The tree synthesis unit is configured to set a value of a flag indicating whether the planar mode corresponding to an axis direction in which no division occurs can be applied to a value indicating that the planar mode cannot be applied when the node is divided into a quadtree or a binary tree.

2. A step of generating a flag indicating whether or not a planar mode is applicable for each axis direction of a node, A point cloud decoding method characterized in that, in the step, when the node is divided into a quadtree or a binary tree, a value of a flag indicating whether or not the planar mode corresponding to an axis direction in which no division occurs is applied is set to a value indicating that the planar mode is not applicable.

3. A program for use in a point group decoding device, comprising: Executing a step configured to generate a flag indicating whether or not the planar mode is applicable for each axis direction of the node; In the above step, when the node is divided into a quadtree or a binary tree, a value of a flag indicating whether or not the planar mode corresponding to an axis direction in which no division occurs is applied is set to a value indicating that the planar mode is not applicable.

Images