Point cloud decoding device, point cloud decoding method, and program

Abstract

To provide compression performance of coding.SOLUTION: In a point cloud decoding device 200, a tree synthesis section 2020 switches processing to be performed according to a sensor type or an operation mode in an Angular mode of Predictive coding. When it is in the Angular mode and the sensor type indicates Non-Spinning LiDAR data or the operation mode indicates the prediction of an elevation angle component and validation of residual decoding, intra-prediction is performed on the basis of values of a parent node and an ancestor node of a node to be processed, for coordinate values r, θ, and φ after polar coordinate conversion of the node to be processed.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a point group decoding device, a point group decoding method, and a program. [Background technology]

[0002] Non-Patent Document 1 discloses a technology for predictive coding in which intra-prediction is performed in an angular mode that takes into account data characteristics and biases for spinning LiDAR data.

[0003] Furthermore, Non-Patent Document 2 discloses a technology for predictive coding in which inter-prediction is performed in an angular mode that takes into account data characteristics and bias for Spinning LiDAR data. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] G-PCC codec description, ISO / IEC JTC1 / SC29 / WG7 N00271 [Non-Patent Document 2] G-PCC 2nd Edition codec description, ISO / IEC JTC1 / SC29 / WG7 N00314 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method of Non-Patent Document 1 has a problem in that compression performance is impaired because intra-prediction is performed on non-spinning LiDAR data without taking into account the characteristics and bias of the data.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a point cloud decoding device, a point cloud decoding method, and a program that can improve the compression performance of encoding. [Means for solving the problem]

[0007] A first feature of the present invention is a point cloud decoding device comprising a tree synthesis unit, which switches the processing to be performed depending on a sensor type or an operation mode in an angular mode of predictive coding, and when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR data or the operation mode indicates enabling of elevation angle component prediction and residual decoding, performs intra prediction for coordinate values ​​r, θ, φ of the node to be processed after polar coordinate transformation based on the values ​​of a parent node and an ancestor node of the node to be processed.

[0008] A second feature of the present invention is a point cloud decoding method comprising the steps of: in an angular mode of predictive coding, switching the processing to be performed depending on a sensor type or an operation mode; and, when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR data or the operation mode indicates enabling of elevation angle component prediction and residual decoding, performing intra prediction for the coordinate values ​​r, θ, and φ of the node to be processed after polar coordinate transformation based on the values ​​of a parent node and an ancestor node of the node to be processed.

[0009] A third feature of the present invention is a program for causing a computer to function as a point cloud decoding device, the point cloud decoding device comprising a tree synthesis unit, the tree synthesis unit switching the processing to be performed depending on a sensor type or an operation mode in an angular mode of predictive coding, and when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR data or the operation mode indicates the enablement of elevation angle component prediction and residual decoding, performing intra prediction for the coordinate values ​​r, θ, φ of the node to be processed after polar coordinate transformation based on the values ​​of a parent node and an ancestor node of the node to be processed. Effect of the Invention

[0010] According to the present invention, it is possible to provide a point group decoding device, a point group decoding method, and a program that can improve the compression performance of encoding. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a point cloud processing system 10 according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of functional blocks of a point group decoding device 200 according to an embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of the configuration of encoded data (bit stream) received by the geometric information decoding unit 2010 of the point cloud decoding device 200 according to an embodiment. [Figure 4] FIG. 4 is a diagram showing an example of the syntax configuration of GPS2011. [Diagram 5] FIG. 5 is a flowchart showing an example of processing in the tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the slice data decoding process in step S505. [Figure 7] FIG. 7 is a diagram for explaining an example of a conversion process into a polar coordinate system, which is one of the coordinate conversions. [Figure 8A] FIG. 8A is a diagram for explaining an example of processing when a flag controlling a sensor type indicates Spinning LiDAR data. [Figure 8B] FIG. 8B is a diagram for explaining an example of processing when the flag controlling the sensor type indicates Non-spinning LiDAR data. [Figure 9] FIG. 9 is a diagram illustrating an example of the prediction method in step S604. [Figure 10] FIG. 10 is a flowchart showing an example of the coordinate prediction process in step S604. [Figure 11] FIG. 11 is a diagram for explaining an example of a process for configuring an intra predictor in the above-mentioned step S1003 when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR or when prediction of the elevation angle component and residual decoding are enabled. [Figure 12] FIG. 12 is a diagram for explaining an example of a process for selecting a predictor from a reference frame in step S1004 when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR or when prediction of the elevation angle component and residual decoding are enabled. [Figure 13] FIG. 13 is a diagram for explaining an example of a process for selecting a predictor from a reference frame in step S1004 when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR or when prediction of the elevation angle component and residual decoding are enabled. [Figure 14] FIG. 14 shows an example of the configuration of encoded data (bit stream) received by the attribute information decoding unit 2060 of the point group decoding device 200 according to an embodiment. [Figure 15] FIG. 15 is an example of the syntax configuration of the APS2611 shown in FIG. [Figure 16] FIG. 16 is a flowchart showing an example of processing by the RAHT unit 2080. [Figure 17] FIG. 17 is a flowchart showing an example of the process of step S28004. [Figure 18] FIG. 18 is a flowchart showing an example of the process of step S28104. [Figure 19] FIG. 19 is a flowchart showing an example of the intra prediction process in step S28112. [Figure 20] FIG. 20 is a diagram showing the relationship between a decoding target node and adjacent nodes in a higher layer. [Figure 21] FIG. 21 is a diagram showing the relationship between a decoding target node and adjacent nodes in the subnode hierarchy. [Figure 22] FIG. 22 is a flowchart showing an example of the intra prediction process in step S28112. [Diagram 23] FIG. 23 is a flowchart showing an example of processing by the RAHT unit 2080. [Figure 24] FIG. 24 is a diagram showing an example of the inter prediction process in step S28111. [Diagram 25] FIG. 25 is a diagram showing an example of functional blocks of the point group encoding device 100 according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0013] (First embodiment) A point cloud processing system 10 according to a first embodiment of the present invention will be described below with reference to Fig. 1 to Fig. 25. Fig. 1 is a diagram showing a point cloud processing system 10 according to the embodiment.

[0014] As shown in FIG. 1, a point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0015] The point cloud encoding device 100 is configured to generate encoded data (bit stream) by encoding an input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bit stream.

[0016] The input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud, such as color information and reflectance of each point.

[0017] Here, such a bit stream may be transmitted from the point group encoding device 100 to the point group decoding device 200 via a transmission path. Also, the bit stream may be stored in a storage medium and then provided from the point group encoding device 100 to the point group decoding device 200.

[0018] (Point Cloud Decoding Device 200) Hereinafter, the point group decoding device 200 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the point group decoding device 200 according to this embodiment.

[0019] As shown in FIG. 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, a LoD calculation unit 2090, an inverse lifting unit 2100, an inverse color transformation unit 2110, and a frame buffer 2120.

[0020] The geometric information decoding unit 2010 is configured to receive as input a bit stream relating to geometric information (geometric information bit stream) out of the bit streams output from the point group encoding device 100, and to decode the syntax.

[0021] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0022] The tree synthesis unit 2020 is configured to receive as input the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating which node in the tree described below the point group exists in, and generate tree information indicating in which area in the space to be decoded the point exists.

[0023] The tree synthesis unit 2020 may be configured to perform the decoding process of the occupancy code within itself.

[0024] This process divides the space to be decoded into rectangles, determines whether a point exists in each rectangle by referring to the occupancy code, divides the rectangle in which the point exists into multiple rectangles, and then generates tree information by recursively repeating the process of referring to the occupancy code.

[0025] Here, when decoding the occupancy code, inter prediction, which will be described later, may be used.

[0026] In this embodiment, a method called "Octree" can be used, which always treats the above-mentioned rectangular parallelepiped as a cube and performs octree division recursively, and a method called "QtBt" can be used, which performs quadtree division and binary tree division in addition to octree division. Whether or not to use "QtBt" is transmitted from the point cloud encoding device 100 as control data.

[0027] Alternatively, when the control data specifies that predictive geometry coding is to be used, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree structure determined in the point cloud encoding device 100.

[0028] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree synthesis unit 2020, and to decode the point cloud based on the approximate surface information.

[0029] Approximate surface information is used when, for example, decoding three-dimensional point cloud data of an object, in cases where the point cloud is densely distributed on the object surface, to approximate the area in which the point cloud exists using a small plane, rather than decoding each individual point cloud.

[0030] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information and decode the point cloud using a method called "Trisoup", for example. A specific processing example of "Trisoup" will be described later. In addition, when decoding a sparse point cloud acquired by Lidar or the like, this processing can be omitted.

[0031] The geometric information reconstruction unit 2040 is configured to reconstruct geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point cloud data to be decoded, based on the tree information generated by the tree synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0032] The inverse coordinate transformation unit 2050 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform the information from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

[0033] The frame buffer 2120 is configured to store, as an input, the geometric information reconstructed by the geometric information reconstruction unit 2040 as a reference frame. When the tree synthesis unit 2020 performs inter-prediction of temporally different frames, the stored reference frame is read out from the frame buffer 2130 and used as the reference frame.

[0034] Here, which reference frame at which time is to be used for each frame may be determined based on control data transmitted from the point group encoding device 100 as a bit stream, for example.

[0035] The attribute information decoding unit 2060 is configured to receive as input a bit stream relating to attribute information (attribute information bit stream) out of the bit streams output from the point group encoding device 100, and to decode the syntax.

[0036] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0037] Moreover, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

[0038] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and a quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, to generate inverse quantized residual information.

[0039] The dequantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090 according to the features of the point group to be decoded. Control data decoded by the attribute information decoding unit 2060 specifies which unit the dequantized residual information is output to.

[0040] The RAHT unit 2080 is configured to receive the inverse quantized residual information generated by the inverse quantization unit 2070 and the geometric information generated by the geometric information reconstruction unit 2040 as input, and to decode the attribute information of each point using a type of Haar transform (inverse Haar transform in the decoding process) called RAHT (Region Adaptive Hierarchical Transform). As a specific process of RAHT, for example, the method described in Non-Patent Document 1 can be used.

[0041] The LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 as an input and generate a Level of Detail (LoD).

[0042] LoD is information for defining a reference relationship (a referencing point and a referenced point) to realize predictive coding, such as predicting attribute information of a certain point from attribute information of another point and encoding or decoding the prediction residual.

[0043] In other words, LoD is information that defines a hierarchical structure in which each point contained in geometric information is classified into multiple levels, and attributes of points belonging to lower levels are encoded or decoded using attribute information of points belonging to higher levels.

[0044] As a specific method for determining the LoD, for example, the method described in Non-Patent Document 1 mentioned above may be used.

[0045] The inverse lifting unit 2100 is configured to decode attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse quantized residual information generated by the inverse quantization unit 2070. As a specific process of inverse lifting, for example, the method described in the above-mentioned Non-Patent Document 1 can be used.

[0046] The inverse color conversion unit 2110 is configured to perform inverse color conversion processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information to be decoded is color information and color conversion has been performed on the point cloud encoding device 100 side. Whether or not to perform such inverse color conversion processing is determined by the control data decoded by the attribute information decoding unit 2060.

[0047] The point group decoding device 200 is configured to decode and output the attribute information of each point in the point group through the above processing.

[0048] (Geometric Information Decoding Part 2010) The control data decoded by the geometric information decoding unit 2010 will be described below with reference to FIGS.

[0049] FIG. 3 shows an example of the structure of the coded data (bit stream) received by the geometric information decoding unit 2010. In FIG.

[0050] First, the bit stream may include a GPS 2011. The GPS 2011 is also called a geometry parameter set, and is a set of control data related to decoding of geometric information. A specific example will be described later. Each GPS 2011 includes at least GPS ID information for identifying each GPS 2011 when there are multiple GPS 2011s.

[0051] Secondly, the bit stream may include GSH2012A / 2012B. GSH2012A / 2012B is also called a geometry slice header or geometry data unit header, and is a set of control data corresponding to a slice described later. In the following description, the term "slice" is used, but slice can also be read as data unit. A specific example will be described later. GSH2012A / 2012B includes at least GPS id information for specifying GPS2011 corresponding to each GSH2012A / 2012B.

[0052] Thirdly, the bit stream may include slice data 2013A / 2013B following the GSH 2012A / 2012B. The slice data 2013A / 2013B includes data in which geometric information is encoded. An example of the slice data 2013A / 2013B is data encoded by occupancy code or predictive coding, which will be described later.

[0053] As described above, the bit stream is configured such that each slice data 2013A / 2013B corresponds to one GSH 2012A / 2012B and one GPS 2011.

[0054] As described above, since the GPS ID information is used to specify which GPS 2011 to refer to in the GSH 2012A / 2012B, a common GPS 2011 can be used for a plurality of slice data 2013A / 2013B.

[0055] In other words, it is not necessary to transmit the GPS 2011 for each slice. For example, as shown in Fig. 3, the bit stream may be configured such that the GPS 2011 is not coded immediately before the GSH 2012B and the slice data 2013B.

[0056] 3 is merely an example. As long as the slice data 2013A / 2013B corresponds to the GSH 2012A / 2012B and the GPS 2011, elements other than those described above may be added as components of the bit stream.

[0057] For example, as shown in Fig. 3, the bitstream may include a sequence parameter set (SPS) 2001. Similarly, when transmitted, the bitstream may be shaped into a configuration different from that shown in Fig. 3. Furthermore, the bitstream may be combined with a bitstream decoded by an attribute information decoding unit 2060 (described later) and transmitted as a single bitstream.

[0058] FIG. 4 is an example of the syntax configuration of GPS2011.

[0059] Note that the syntax names described below are merely examples. If the syntax functions described below are similar, the syntax names may be different.

[0060] The GPS 2011 may include GPS ID information (gps_geom_parameter_set_id) for identifying each GPS 2011.

[0061] The Descriptor column in Fig. 4 indicates how each syntax is coded. ue(v) indicates an unsigned zeroth-order exponential Golomb code, and u(1) indicates a 1-bit flag.

[0062] The GPS 2011 may include a flag (geom_tree_type) for controlling the tree type in the tree synthesis unit 2020 .

[0063] For example, if the value of geom_tree_type is "1", it may be defined that predictive geometry coding is used, and if the value of geom_tree_type is "0", it may be defined that Octree is used.

[0064] The GPS 2011 may include a flag (interprediction_enabled_flag) that controls whether or not the tree synthesis unit 2020 performs inter prediction.

[0065] For example, when the value of interprediction_enabled_flag is "0", it may be defined that inter prediction is not performed, and when the value of interprediction_enabled_flag is "1", it may be defined that inter prediction is performed.

[0066] Note that the interpretation_enabled_flag may be included in the SPS2001 instead of the GPS2011.

[0067] The GPS 2011 may include a flag (geom_tree_type) for controlling the tree type in the tree synthesis unit 2020. For example, when the value of geom_tree_type is "1", it may be defined that predicitive coding is used, and when the value of geom_tree_type is "0", it may be defined that predicitive coding is not used.

[0068] Note that geom_tree_type may be included in SPS2001 instead of GPS2011.

[0069] The GPS 2011 may include a flag (geom_angular_enabled) for controlling whether or not the tree synthesis unit 2020 performs processing in angular mode.

[0070] For example, when the value of geom_angular_enabled is “1”, it may be defined that predictive coding is performed as the angular mode, and when the value of geom_angular_enabled is “0”, it may be defined that predictive coding is not performed as the angular mode.

[0071] Note that geom_angular_enabled may be included in SPS2001 instead of GPS2011.

[0072] The GPS 2011 may include a flag (sensor_type) for controlling the sensor type in the tree composition unit 2020 in the angular mode.

[0073] For example, it may be defined that when the value of sensor_type is "1", the data is processed as spinning LiDAR data, and when the value of sensor_type is "0", the data is processed as non-spinning LiDAR data.

[0074] Note that sensor_type may be included in SPS2001 instead of GPS2011. Also, instead of sensor_type, a flag for controlling switching of operation modes in Angular mode, which will be described later, may be used.

[0075] The GPS 2011 may include a flag (global_motion_enabled_flag) that controls whether or not the tree synthesis unit 2020 performs global motion compensation for inter prediction.

[0076] For example, when the value of global_motion_enabled_flag is "0", it may be defined that global motion compensation is not performed, and when the value of global_motion_enabled_flag is "1", it may be defined that global motion compensation is performed.

[0077] When global motion compensation is performed, each slice data may include a global motion vector.

[0078] Note that global_motion_enabled_flag may be included in SPS2001 instead of GPS2011.

[0079] (Tree Synthesis Department 2020) The processing of the tree composing unit 2020 will be described below with reference to Fig. 5 to Fig. 13. Fig. 5 is a flowchart showing an example of the processing in the tree composing unit 2020. Note that, below, an example will be described in which trees are composed using "Predictive geometry coding".

[0080] In addition, terms such as "Predictive geometry," "Predictive geometry coding," and "Predictive tree" may be used instead of "Predictive geometry coding."

[0081] As shown in FIG. 5, in step S501, the tree synthesis unit 2020 determines whether to use inter prediction based on the value of interprediction_enabled_flag.

[0082] If the tree synthesis unit 2020 determines that inter prediction is to be used, the process proceeds to step S502, and if the tree synthesis unit 2020 determines that inter prediction is not to be used, the process proceeds to step S505.

[0083] In step S502, the tree synthesis unit 2020 obtains a reference frame from the frame buffer 2120. The frame buffer 2120 may store one previously decoded frame, and a decoded frame may be added to the frame buffer 2020 each time the decoding of one frame or a specified number of frames is completed. After obtaining the reference frame, the tree synthesis unit 2020 proceeds to step S503.

[0084] In step S503, the tree synthesis unit 2020 determines whether or not to perform global motion compensation based on the global_motion_enabled_flag.

[0085] If the tree synthesis unit 2020 determines that global motion compensation is to be performed, the process proceeds to step S504, and if it determines that global motion compensation is not to be performed, the process proceeds to step S505.

[0086] In step S504, the tree synthesis unit 2020 performs global motion compensation on the reference frame obtained in step S502.

[0087] Here, global motion compensation is a process for correcting the global position shift for each frame, and applies rotation and translation based on the global motion vector decoded by the geometric information decoding unit 2010 to the entire reference frame or to a group of points within a specified range.

[0088] After performing global motion compensation, the tree synthesis unit 2020 proceeds to step S505.

[0089] In step S505, the tree composing unit 2020 decodes the slice data. Specific processing in step S505 will be described later. After decoding the slice data, the tree composing unit 2020 proceeds to step S506.

[0090] In step S506, the tree synthesis unit 2020 ends the process. Note that the processes in steps S503 and S504, that is, the determination and execution of global motion compensation, may be performed in the decoding process of slice data in step S505.

[0091] FIG. 6 is a flowchart showing an example of the slice data decoding process in step S505.

[0092] As shown in FIG. 6, in step S601, the tree synthesis unit 2020 constructs a prediction tree corresponding to slice data.

[0093] The slice data may include a depth-first list of the number of children for each node in the prediction tree. The prediction tree may be constructed by starting with the root node and adding to each node, in depth-first order, the number of children specified in the list.

[0094] After completing construction of the prediction tree, the tree synthesis unit 2020 proceeds to step S602.

[0095] In step S602, the tree synthesis unit 2020 determines whether or not processing of all nodes of the prediction tree has been completed.

[0096] If the tree synthesis unit 2020 determines that the processing of all nodes of the prediction tree has been completed, the process proceeds to step S607, and if it determines that the processing of all nodes of the prediction tree has not been completed, the process proceeds to step S603.

[0097] In step S603, the tree synthesis unit 2020 selects a node to be processed from the prediction tree.

[0098] The tree merging unit 2020 may select, as the processing target node, the node that is next to the previous processing target node in depth-first order.

[0099] After completing the selection of the node to be processed, the tree merging unit 2020 proceeds to step S604.

[0100] In step S604, the tree merge unit 2020 predicts the coordinates of the point corresponding to the node to be processed. A specific method for predicting the coordinates will be described later.

[0101] After completing this prediction, the tree synthesis unit 2020 proceeds to step S605.

[0102] In step S605, the tree synthesis unit 2020 decodes the prediction residual of the coordinates of the point corresponding to the node to be processed. The slice data may include the prediction residual of the coordinates of the point corresponding to each node.

[0103] After completing the decoding of the prediction residual of the node to be processed, the tree synthesis unit 2020 proceeds to step S606.

[0104] In step S606, the tree synthesis unit 2020 reconstructs the coordinates of the point corresponding to the node to be processed. The tree synthesis unit 2020 may obtain the coordinates of the point by adding the coordinates predicted in step S604 and the residual decoded in step S605.

[0105] After the tree synthesis unit 2020 completes the reconstruction of the coordinates, it returns to step 602 .

[0106] In step S607, the tree synthesis unit 2020 ends the process of step S505.

[0107] Here, the order of steps S604 and S605 may be reversed.

[0108] When the Angular mode is used, the tree synthesis unit 2020 may take into consideration that the coordinate values ​​handled in steps S604 to S606 are values ​​after coordinate transformation, and step S606 may include the inverse transformation process.

[0109] Fig. 7 is a diagram for explaining an example of a conversion process to a polar coordinate system, which is one of the coordinate conversions. As shown in Fig. 7, the coordinate values ​​of each point are represented by a radius r, an elevation angle θ, and an azimuth angle φ.

[0110] Furthermore, among the coordinate values ​​after the polar coordinate conversion, the elevation angle θ and the azimuth angle φ may be further converted appropriately. For example, the two values ​​of the elevation angle θ and the azimuth angle φ may be treated as two-dimensional orthogonal coordinates, and a two-dimensional polar coordinate conversion may be further applied.

[0111] FIG. 8A is a diagram illustrating an example of processing when a flag controlling a sensor type indicates spinning LiDAR data, and FIG. 8B is a diagram illustrating an example of processing when a flag controlling a sensor type indicates non-spinning LiDAR data.

[0112] As shown in FIG. 8A, when the sensor type indicates a spinning LiDAR, in steps S604 and S605, the tree synthesis unit 2020 processes only the radius r and the azimuth angle φ among the coordinate values, and in step S606, the radius r and the azimuth angle φ are reconstructed, and the elevation angle θ may be set from a predefined fixed value.

[0113] For example, the tree synthesis unit 2020 may set the elevation angle θ of one of multiple lasers in the LiDAR, and a laser ID indicating which laser it is may be included in the slice data.

[0114] On the other hand, when the sensor type indicates Non-Spinning LiDAR, the tree synthesis unit 2020 may perform prediction, residual decoding, and coordinate reconstruction processing for all of the coordinate values ​​r, θ, and φ in steps S604 to S606.

[0115] The tree synthesis unit 2020 may switch between the above two operations based on the value of a flag that controls whether to enable prediction and residual decoding of the elevation angle component, rather than on the flag that controls the sensor type.

[0116] FIG. 9 is a diagram illustrating an example of the prediction method in step S604.

[0117] In step S604, the tree synthesis unit 2020 may perform prediction using only one point out of the N consecutive points as shown in Fig. 9. Then, the tree synthesis unit 2020 may share the predicted value of the one point as the predicted values ​​of the remaining N-1 points.

[0118] FIG. 10 is a flowchart showing an example of the coordinate prediction process in step S604.

[0119] As shown in FIG. 10, in step S1001, the tree synthesis unit 2020 decodes the predictor flag.

[0120] Here, the slice data may include a flag indicating a predictor to be used for each node. For example, the slice data may include a flag indicating whether the predictor is an inter predictor or an intra predictor, an index of the inter predictor, or the like, which is similar to the contents described in Non-Patent Documents 1 and 2. The slice data may also include other flags, which will be described later.

[0121] After decoding the predictor flag, the tree synthesis unit 2020 proceeds to step S1002.

[0122] In step S1002, the tree synthesis unit 2020 determines whether to use an inter predictor based on the predictor flag decoded in step S1001.

[0123] If the tree synthesis unit 2020 determines that an inter predictor is to be used, the process proceeds to step S1004, and if it determines that an inter predictor is not to be used, the process proceeds to step S1003.

[0124] In step S1003, the tree synthesis unit 2020 performs intra prediction on the coordinates of the node to be processed.

[0125] When performing intra prediction, the tree synthesis unit 2020 configures a predictor based on the coordinates of the parent node or ancestor node (for example, the parent node of the parent node) of the node to be processed, and predicts the coordinates of the node to be processed.

[0126] When the tree synthesis unit 2020 is in the angular mode and the sensor type is a spinning LiDAR or the prediction and residual decoding of the elevation angle component are disabled, the tree synthesis unit 2020 may use the methods described in Non-Patent Documents 1 and 2 as a method for configuring an intra predictor.

[0127] The tree synthesis unit 2020 may use a configuration method of an intra predictor, which will be described later, when the angular mode is selected and the sensor type is a non-spinning LiDAR or when prediction of the elevation angle component and residual decoding are enabled.

[0128] After completing the intra prediction, the tree synthesis unit 2020 proceeds to step S1005.

[0129] In step S1004, the tree synthesis unit 2020 performs inter prediction on the coordinates of the node to be processed.

[0130] When performing inter prediction, the tree synthesis unit 2020 selects a node corresponding to the node to be processed from the reference frame as a predictor, and sets the coordinates of the selected predictor as the predicted value of the coordinates of the node to be processed.

[0131] When the tree synthesis unit 2020 is in angular mode and the sensor type indicates spinning LiDAR or when prediction and residual decoding of the elevation angle component are disabled, the tree synthesis unit 2020 may select a predictor from a reference frame using a method similar to that described in Non-Patent Document 2.

[0132] In addition, when the mode is angular and the sensor type indicates non-spinning LiDAR or when prediction and residual decoding of the elevation angle component are enabled, the tree synthesis unit 2020 selects a predictor from a reference frame, as described below.

[0133] After completing the inter prediction, the tree synthesis unit 2020 proceeds to step S1005.

[0134] In step S1005, the tree synthesis unit 2020 ends the process of step S604.

[0135] FIG. 11 is a diagram for explaining an example of a process for configuring an intra predictor in the above-mentioned step S1003 when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR or when prediction of the elevation angle component and residual decoding are enabled. The intra predictor predicts the coordinate value q=(r, θ, φ) of the node being processed. · The intra predictor may predict coordinate value q=0 as no prediction. The intra predictor may predict q=qk-1 using the coordinate value qk-1 of the parent node. The intra predictor may use the coordinate value qk-1 of the parent node and the coordinate value qk-2 of the grandparent node, and set q=qk-1(qk-1-qk-2). In a prediction tree, if the average absolute distance between the coordinate values ​​of adjacent nodes in the most recently processed branch is Δq, the intra predictor may predict q=qk-1+Δq using the coordinate value qk-1 of the parent node. Among the nodes belonging to the most recently processed branch in the prediction tree, the node most similar to the parent node of the node being processed is selected, and when the absolute distance in coordinate values ​​between this similar node and its child node is Δq', the intra predictor may predict q=qk-1+Δq' using the coordinate value qk-1 of the parent node.

[0136] The tree synthesis unit 2020 may decode a flag indicating which of these intra predictors to use for each node. The flag may be included in the slice data.

[0137] 12 and 13 are diagrams for explaining an example of a process of selecting a predictor from a reference frame in the above-mentioned step S1004 when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR or when prediction of the elevation angle component and residual decoding are enabled.

[0138] In the example of FIG. 12, the tree synthesis unit 2020 searches for a node in the reference frame whose coordinate values ​​θ and φ are closest to the parent node of the node to be processed, and selects a predictor from its child node or grandchild node.

[0139] In the example of FIG. 13, the tree synthesis unit 2020 assumes that each node has a unique node ID within a frame, and selects a predictor from a reference frame based on the node ID of the node to be processed.

[0140] The tree synthesis unit 2020 may be configured such that, in the angular mode and when the sensor type indicates Non-Spinning LiDAR or when elevation component prediction and residual decoding are enabled, the prediction tree is configured so that N serial branches are connected from the root, and each node may hold a branch ID indicating the branch to which it belongs.

[0141] The tree synthesis unit 2020 may limit the range in which to search for a predictor from the reference frame, based on the branch ID of the parent node of the node to be processed.

[0142] (Attribute information decoding unit 2060) The control data decoded by the attribute information decoding unit 2060 will be described below with reference to FIGS.

[0143] FIG. 14 shows an example of the structure of the coded data (bit stream) received by the attribute information decoding unit 2060. In FIG.

[0144] FIG. 14 is an example of the syntax configuration of the APS2611.

[0145] Note that the syntax names described below are merely examples. If the syntax functions described below are similar, the syntax names may be different.

[0146] The APS 2611 may include APS id information (aps_geom_parameter_set_id) for identifying each APS 2611.

[0147] The Descriptor column in Fig. 15 indicates how each syntax element is coded. ue(v) indicates an unsigned zeroth-order exponential Golomb code, and u(1) indicates a 1-bit flag.

[0148] The APS 2611 may include a flag (attr_coding_type) for controlling whether the inverse quantization unit 2070 outputs the inverse quantized residual information to the RAHT unit 2080 or the LoD calculation unit 2090.

[0149] For example, it may be defined that when the value of attr_coding_type is “1”, the data is output to the LoD calculation unit 2090 , and when the value of attr_coding_type is “0”, the data is output to the RAHT unit 2080 .

[0150] The APS 2611 may include a flag (raht_prediction_enabled) for controlling whether or not to perform prediction of attribute information in the RAHT unit 2080.

[0151] For example, when the value of raht_prediction_enabled is "1", it may be defined that prediction of attribute information is performed, and when the value of raht_prediction_enabled is "0", it may be defined that prediction of attribute information is not performed.

[0152] The APS 2611 may include a flag (raht_subnode_prediction_enable_flag) that controls whether or not the RAHT unit 2080 uses subnodes to predict attribute information.

[0153] For example, when the value of raht_subnode_prediction_enable_flag is "1", it may be defined that subnodes are used to predict attribute information, and when the value of raht_subnode_prediction_enable_flag is "0", it may be defined that subnodes are not used to predict attribute information.

[0154] The APS 2611 may include weight parameters (raht_prediction_weights) used when the RAHT unit 2080 performs intra prediction of attribute information.

[0155] For example, the value of raht_prediction_weights may be defined depending on the type of adjacent node used for intra prediction.

[0156] The APS 2611 may include a flag (raht_inter_prediction_enabled) for controlling whether or not to perform inter prediction of attribute information in the RAHT unit 2080.

[0157] For example, when the value of raht_inter_prediction_enabled is "1", it may be defined that prediction of attribute information is performed, and when the value of raht_inter_prediction_enabled is "0", it may be defined that prediction of attribute information is not performed.

[0158] The APS 2611 may include a value (raht_inter_prediction_depth_minus1) indicating a layer at which inter prediction of attribute information is enabled in the RAHT unit 2080.

[0159] For example, when the value of raht_inter_prediction_depth_minus1 is "N-1", inter prediction may be enabled in up to the top N layers of the Octree structure.

[0160] (RAHT Division 2080) An example of the processing of the RAHT unit 2080 will be described with reference to FIGS.

[0161] FIG. 16 is a flowchart showing an example of processing by the RAHT unit 2080.

[0162] 16, in step S28001, the RAHT unit 2080 recursively divides the nodes into octrees until a predetermined size is reached, using a method called Octree. After the division is completed, the operation proceeds to step S28002.

[0163] In step S28002, the RAHT unit 2080 counts up the total number of points belonging to the lower layer of each node divided by the octree.

[0164] Specifically, the RAHT unit 2080 scans the nodes of a certain layer in order and records the number of points belonging to each node. Next, the RAHT unit 2080 adds up the numbers of points recorded in the child nodes of each node in the node one layer above to calculate the number of points belonging to each node.

[0165] The RAHT unit 2080 repeats the above scanning from the lowest layer to the highest layer. The total number of acquired points is used as a weight for the inverse transformation of the RAHT in step S28005 described later. After this calculation is completed, the operation proceeds to step S28003.

[0166] In step S28003, the RAHT unit 2080 decodes the DC coefficient of the node belonging to the highest layer of the Octree. Alternatively, the RAHT unit 2080 may calculate the DC coefficient by predicting the DC coefficient using intra prediction and decoding and adding up the prediction residual of the DC coefficient.

[0167] After completing the decoding of the DC coefficients, the RAHT unit 2080 calculates the total number of points belonging to the root node acquired in step S28002, w root and the decoded DC coefficient DC root Using this, the attribute value A of the root node root Calculate.

[0168]

number

[0169] In step S28004, the RAHT unit 2080 determines whether or not decoding of the attribute information of all nodes included in the layer has been completed.

[0170] If not completed, operation proceeds to step S28005; if completed, operation proceeds to step S28007.

[0171] In step S28005, the RAHT unit 2080 decodes the AC coefficients. The details will be described later. After the decoding is completed, the operation proceeds to step S28006.

[0172] In step S28006, the RAHT unit 2080 calculates attribute values ​​using the inverse transform of the RAHT based on the total number of points belonging to the lower hierarchy of each node, the decoded AC coefficients, and the DC coefficients calculated from the nodes in the upper hierarchy using the method described below.

[0173] Here, the inverse transformation of RAHT is performed in units of 8 nodes (2 x 2 x 2) divided by the Octree.

[0174] Specifically, attribute values ​​A1, A2, … A k is the DC coefficient DC of a node that holds k subnodes, and the AC coefficients AC1,AC2,…AC k-1 and the total number of points belonging to the lower hierarchy of each subnode, w=w1, w2, … w k Using this, it can be calculated using the following formula (1).

[0175]

number

[0176] It is assumed that the conversion process is performed repeatedly from the higher-level nodes to the lower-level nodes,

[0177]

number

[0178] In step S28007, the RAHT unit 2080 determines whether processing of all layers is complete. If not, the operation moves the processing target layer to the next lower layer and proceeds to step S28004. If processing is complete, the operation proceeds to step S28008 and ends the processing.

[0179] FIG. 17 is a flowchart showing an example of the process of step S28005.

[0180] 17, in step S28101, the RAHT unit 2080 determines whether to predict AC coefficients. When making such a determination, the RAHT unit 2080 may refer to raht_prediction_enabled and use the value thereof.

[0181] The RAHT unit 2080 may decode a flag indicating whether or not to perform prediction of an AC coefficient in the currently processed node, and use the value of the flag.

[0182] Such flags are: It may be decoded coefficient by coefficient, The flag may be decoded on a per-node basis or a per-layer basis. The flag may be decoded only if the value of raht_prediction_enabled is "1", which indicates that prediction is enabled. The flag may be included in the slice data.

[0183] If the result of the determination is that the AC coefficients are not to be predicted, the operation proceeds to step S28102, and if the result is that the AC coefficients are to be predicted, the operation proceeds to steps S28103 and S28104.

[0184] In step S28102, the RAHT unit 2080 decodes the AC coefficients. After the decoding is completed, the operation proceeds to step S28106, and the process ends.

[0185] In step S28103, the RAHT 2080 decodes the AC coefficient residuals. After the decoding is completed, the operation proceeds to step S28105.

[0186] In step S28104, the RAHT unit 2080 predicts AC coefficients. Inter prediction or intra prediction may be used for predicting the AC coefficients.

[0187] The RAHT unit 2080 may first predict the attribute values, and then calculate the predicted values ​​of the AC coefficients by the RAHT. This will be described in detail later. After the prediction of the AC coefficients is completed, the operation proceeds to step S28105.

[0188] In step S28105, the RAHT unit 2080 adds the residual of the decoded AC coefficients to the predicted AC coefficients to reconstruct the AC coefficients. After the reconstruction is completed, the operation proceeds to step S28106, and the process ends.

[0189] FIG. 18 is a flowchart showing an example of the process of step S28104.

[0190] As shown in Fig. 18, in step S28107, the RAHT unit 2080 determines whether inter prediction is enabled. The RAHT unit 2080 may refer to raht_inter_prediction_enabled for the determination and use the value thereof. If the result of the determination is that inter prediction is enabled, this operation proceeds to step S28109, and if inter prediction is disabled, this operation proceeds to step S28112.

[0191] In step S28109, the RAHT unit 2080 determines whether the depth of the layer including the node to be processed is equal to or less than a threshold. The RAHT unit 2080 may refer to raht_inter_prediction_depth_minus1 as the threshold and use that value.

[0192] If the result of the determination is that the depth is equal to or less than the threshold, the operation proceeds to step S28110, and if the depth is greater than the threshold, the operation proceeds to step S28112.

[0193] In step S28110, the RAHT unit 2080 determines whether or not to perform inter prediction on the AC coefficient of the node to be processed.

[0194] For this determination, the RAHT unit 2080 may check whether inter prediction is possible, and perform inter prediction if possible, and may not perform inter prediction if not possible.

[0195] For the determination, the RAHT unit 2080 may decode a flag indicating whether or not to perform inter prediction on the AC coefficient of the node to be processed, and use the value of this flag.

[0196] The flag may be set to indicate the smaller of the costs of intra prediction and inter prediction during encoding. The cost may be calculated by taking into account one or a combination of the magnitude of the residual, the distortion of the reconstructed value, and the estimated code amount.

[0197] Such a flag may be decoded for each coefficient, for each node, for each set of multiple nodes, or for each layer.

[0198] Such a flag may be decoded and a determination may be made only if it is determined that inter prediction is feasible.

[0199] Such a flag may be included in the slice data.

[0200] Such flags may be coded using arithmetic coding, in which case the context referenced in the arithmetic coding may be one context per frame or one context per layer.

[0201] In step S28111, the RAHT unit 2080 performs inter prediction of the AC coefficients of the node to be processed. This will be described in detail later.

[0202] In step S28112, the RAHT unit 2080 performs intra prediction of the AC coefficients of the node to be processed. This will be described in detail later.

[0203] In step S28113, the process of step S28104 ends. Note that the conditional branch in step S28109 may be omitted.

[0204] In the inter prediction process in step S28111, a process equivalent to the intra prediction process in step S28112 may also be performed, and prediction may be performed by combining the results of inter prediction and intra prediction.

[0205] FIG. 19 is a flowchart showing an example of the intra prediction process in step S28112.

[0206] 19, in step S28201, the RAHT unit 2080 determines whether or not to perform intra prediction using adjacent nodes in the subnode hierarchy. The RAHT unit 2080 may refer to raht_subnode_prediction_enable_flag and use the value thereof for the determination.

[0207] When the RAHT unit 2080 does not use adjacent nodes in the subnode hierarchy, it performs intra prediction using only adjacent nodes in a higher hierarchy.

[0208] In this case, the adjacent nodes in the higher hierarchy are the six nodes adjacent to the parent node of the node to be decoded on the face side, the 12 nodes adjacent to the edge side, and the parent node itself, which are a total of 19 nodes, three nodes adjacent to the face side of the node to be decoded on the face side, three nodes adjacent to the edge side, and the seven nodes of the parent node itself.

[0209] FIG. 20 is a diagram showing the relationship between a decoding target node and adjacent nodes in a higher layer.

[0210] When using adjacent nodes in the subnode hierarchy, the RAHT unit 2080 performs intra prediction using adjacent nodes in a higher hierarchy and adjacent nodes in the subnode hierarchy.

[0211] Here, an adjacent node in the subnode hierarchy is a subnode of an adjacent node in a higher hierarchy that has a face or edge adjacent to the decoding target node and has already been decoded.

[0212] FIG. 21 is a diagram showing the relationship between a decoding target node and adjacent nodes in the subnode hierarchy.

[0213] If the determination result is that intra prediction is to be performed without using adjacent nodes in the subnode hierarchy, this operation proceeds to step S28202, and if intra prediction is to be performed using adjacent nodes in the subnode hierarchy, this operation proceeds to step S28204.

[0214] In step S28202, the RAHT unit 2080 acquires the attribute value of the adjacent node in the upper layer. After acquiring the attribute value of the adjacent node in the upper layer, this operation proceeds to step S28203.

[0215] In step S28203, the RAHT unit 2080 predicts the attribute value of the node to be decoded.

[0216] The RAHT unit 2080 obtains the attribute values ​​attr i and the weight w according to the type of adjacent node i i Using these, the attribute value attr may be predicted using the following formula:

[0217]

number

[0218] After the prediction of the attribute value is completed, the operation proceeds to step S28207.

[0219] In step S28204, the RAHT unit 2080 acquires the attribute value of the adjacent node in the upper layer.

[0220] Here, the targets for which attribute values ​​are to be obtained are adjacent nodes in a higher hierarchy whose subnodes have not yet been decoded, or adjacent nodes in a higher hierarchy whose subnodes have been decoded but whose subnodes do not have any subnodes adjacent to the node to be decoded via faces or edges.

[0221] After the attribute value has been acquired, the operation proceeds to step S28205.

[0222] In step S28205, the RAHT unit 2080 acquires the attribute value of the adjacent node in the subnode hierarchy. After acquiring the attribute value of the adjacent node in the subnode hierarchy, this operation proceeds to step S28206.

[0223] In step S28206, the RAHT unit 2080 predicts the attribute value of the node to be decoded.

[0224] The RAHT unit 2080 obtains the attribute values ​​attr i and the weight w according to the type of adjacent node i i Using these, the attribute value attr may be predicted using the following formula:

[0225]

number

[0226] After the attribute value prediction is completed, the operation proceeds to step S28207.

[0227] In step S28207, the RAHT unit 2080 converts the predicted attribute values ​​into AC coefficients. The AC coefficients are generated by performing RAHT on the predicted attribute values. For example, the RAHT unit 2080 may use the method described in Non-Patent Document 1 as such a conversion method.

[0228] The above describes an example in which the RAHT unit 2080 uses the attribute values ​​predicted in step S28206 directly to convert the AC coefficients in step S28207, but the RAHT unit 2080 may smooth the predicted attribute values ​​and then convert the AC coefficients.

[0229] For example, as shown in FIG. 22, after predicting the attribute value, the RAHT unit 2080 may determine whether or not to perform smoothing in step S1301.

[0230] In making such a determination, the RAHT unit 2080 may refer to raht_smoothing_enable_flag and use the value thereof.

[0231] If smoothing is to be performed, the operation proceeds to step S1302. If smoothing is not to be performed, the operation proceeds to step S28207.

[0232] In step S1302, the RAHT unit 2080 may smooth the attribute values.

[0233] For example, the RAHT unit 2080 calculates the smoothed attribute value Attr smoothing The predicted attribute value Attr i and weight α i Alternatively, the weighted average may be calculated as follows:

[0234]

number

[0235] In addition, the RAHT unit 2080 uses the weight α i A hard-coded value may be used as the weights, or the value may be referenced and used as the weights.

[0236] In addition, the RAHT unit 2080 performs, for example, the smoothed attribute value Attr smoothing The predicted value Attr0 of the decoded node itself, the predicted attribute value Attr of the subnode i other than the decoded node among the subnodes in the same parent node as the decoded node, i , weight β i and a threshold value Thr, may be used to perform clipping as follows:

[0237]

number

[0238] The clipping function Clip3 is

[0239]

number

[0240] Here, for the target subnode i, the RAHT unit 2080 may treat the decode target node as a face-adjacent node, a face-adjacent node and an edge-adjacent node, or all subnodes within the same parent node.

[0241] In addition, the RAHT unit 2080 uses the weight β i A hard-coded value may be used as the weights, or the value may be referenced and used as the raht_smoothing_clipping_weights.

[0242] Furthermore, the RAHT unit 2080 may use a hard-coded value as the threshold value Thr, or may refer to raht_smoothing_clipping_threshold and use that value.

[0243] In the above, an example has been described in which the RAHT unit 2080 decodes AC coefficients of both the color difference signal and the luminance signal, but the RAHT unit 2080 may skip decoding AC coefficients of the color difference signal only in the bottom layer of the octree.

[0244] For example, as shown in FIG. 23, in step S1401, the RAHT unit 2080 may determine whether or not to skip decoding of AC coefficients of chrominance signals only in the bottom layer of the octree.

[0245] If skipping, the operation proceeds to step S1402, otherwise the operation proceeds to step S28004.

[0246] In step S1402, the RAHT unit 2080 determines whether the node to be decoded is at the bottom layer of the octree.

[0247] If it is the bottom layer, the operation proceeds to step S1403. If it is not the bottom layer, the operation proceeds to step S28004.

[0248] In step S1403, the RAHT unit 2080 decodes the AC coefficients other than the color difference signals.

[0249] The RAHT unit 2080 performs the same process as in step S28004 for decoding AC coefficients other than those of the color difference signals, sets the AC coefficients of the color difference signals to 0, and calculates attribute values ​​in the subsequent step S28005.

[0250] After the decoding of AC coefficients other than the color difference signal is completed, the operation proceeds to step S28006.

[0251] FIG. 24 is a diagram showing an example of the inter prediction process in step S28111.

[0252] The RAHT unit 2080 predicts the AC coefficients of the processing target node using information of a reference node, which is a corresponding node in a reference frame. Here, the information of the reference node may be its attribute value or AC coefficient. The reference frame may indicate another decoded frame, and the information may be included in the previous frame buffer 2120.

[0253] The RAHT unit 2080 may apply the same octree structure as the processing target frame to the reference frame. In such a case, a node may be set at a position where there is no point. Such a node is called an empty node. If the reference node is an empty node, the RAHT unit 2080 may disable inter prediction in step S28110.

[0254] The RAHT unit 2080 may apply an Octree to the reference frame independently of the processing target frame, and set an Octree structure different from that of the processing target frame. In such a case, a node may not necessarily exist at the same position as that of the processing target frame. If a reference node is not found at a position corresponding to the processing target node, the RAHT unit 2080 may disable inter prediction in step S28143.

[0255] If the reference node is an empty node, or if the reference node cannot be found, the RAHT unit 2080 may estimate and interpolate the information of the reference node using information of nodes in nearby positions within the reference frame.

[0256] For example, the RAHT unit 2080 may estimate and interpolate the average value of the attribute values ​​or AC coefficients of adjacent nodes, nearest neighbor nodes, or k-nearest neighbor nodes with respect to the reference node position as the attribute value or AC coefficient of the reference node, respectively.

[0257] The RAHT unit 2080 may predict the AC coefficients of the processing target node from, for example, the attribute values ​​of the reference node.

[0258] Specifically, the RAHT unit 2080 converts the decoded attribute value Attr inter The predicted value Attr of the attribute value of the node to be processed is calculated using pred The predicted value Attr pred By applying RAHT to the target node, the predicted value AC pred It is also possible to ask for

[0259] Attr pred =Attr inter AC pred =RAHT(Attr pred ) The RAHT unit 2080 may predict the AC coefficients of the processing target node directly from the AC coefficients of the reference node, for example.

[0260] Specifically, the RAHT unit 2080 calculates the AC coefficient values ​​AC of the reference nodes using the RAHT in the reference frame. inter The calculated value is used as the predicted value AC pred It is also possible to use the following.

[0261] AC pred =AC inter The RAHT unit 2080 may obtain the AC coefficients of the reference node by recording the AC coefficients of each node of the reference frame in the frame buffer 2120 and referring to the values ​​in the frame buffer 2120. In this case, if there are no AC coefficients of the reference node in the frame buffer 2120, the RAHT unit 2080 may determine in step S28110 that inter prediction is not executable.

[0262] In addition, RAHT section 2080 is Attr inter and A.C. inter may be multiplied by a scaling factor α.

[0263] Attr pred =αAttr inter or AC pred = αAC inter The coefficient α may be any real number. The coefficient α may be decoded for each node or for each layer. The coefficient α may be included in the slice data.

[0264] For example, the coefficient α may be defined using the hierarchical depth depth as follows, and α′ may be decoded instead of the coefficient α.

[0265] α=1+α'·2 -depth The RAHT unit 2080 may perform a similar operation in the inter prediction of the DC coefficient in step S28003.

[0266] DC pred = αDC inter Here, we use the DC coefficient of the reference node as DC inter The predicted value of the DC coefficient of the root node is DC pred Let us assume that.

[0267] Furthermore, the RAHT unit 2080 may calculate predicted values ​​of attribute values ​​or AC coefficients by combining inter prediction and intra prediction.

[0268] For example, an example of the case where the RAHT unit 2080 requests prediction of an attribute value is shown below.

[0269] Attr pred =W inter ·Attr inter +W intra ·Attr intra Here, Attr inter and Attr intra are the inter-prediction and intra-prediction of the attribute value, respectively. Also, W inter and W intra are the weights of the inter-prediction and intra-prediction, respectively. W inter and W intra may be determined such that the intra-prediction is more emphasized for deeper layers according to the depth depth of the processing target layer. For example, W inter =1-depth / N W intra =depth / N N is the maximum value of the depth of the layer where the inter-prediction is effective. The combination of the inter-prediction and intra-prediction may be effective only at a specific layer. For example, the combination of the inter-prediction and intra-prediction may be effective only when M < depth < N. M is an arbitrary real number less than N and may be decoded as header information such as APS.

[0270] (Point cloud encoding device 100) Hereinafter, with reference to FIG. 25, the point cloud encoding device 100 according to the present embodiment will be described. FIG. 25 is a diagram showing an example of the functional blocks of the point cloud encoding device 100 according to the present embodiment.

[0271] As shown in FIG. 25, the point cloud encoding device 100 includes a coordinate transformation unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information encoding unit 1050, a geometric information reconstruction unit 1060, a color conversion unit 1070, an attribute transfer unit 1080, a RAHT unit 1090, a LoD calculation unit 1100, a lifting unit 1110, an attribute information quantization unit 1120, an attribute information encoding unit 1130, and a frame buffer 1140.

[0272] The coordinate conversion unit 1010 is configured to convert the three-dimensional coordinate system of the input point cloud into any different coordinate system. The coordinate conversion may convert the x, y, and z coordinates of the input point cloud into any s, t, and u coordinates by rotating the input point cloud, for example. As a variation of the conversion, the coordinate system of the input point cloud may be used as it is.

[0273] The geometric information quantization unit 1020 is configured to quantize the position information of the input point group after the coordinate transformation and to remove points with overlapping coordinates. When the quantization step size is 1, the position information of the input point group and the position information after quantization match. In other words, when the quantization step size is 1, it is equivalent to the case where quantization is not performed.

[0274] The tree analysis unit 1030 is configured to receive position information of the quantized point group as input, and to generate an occupancy code indicating at which node in the encoding target space a point exists, based on a tree structure described below.

[0275] In this process, the tree analysis unit 1030 is configured to recursively divide the encoding target space into rectangular parallelepipeds to generate a tree structure.

[0276] If a point exists within a certain rectangular parallelepiped, a tree structure can be generated by recursively dividing the rectangular parallelepiped into multiple rectangular parallelepipeds until the rectangular parallelepiped reaches a specified size. Each such rectangular parallelepiped is called a node. Each rectangular parallelepiped generated by dividing a node is called a child node, and the occupancy code is expressed as 0 or 1 to indicate whether or not a point is included in the child node.

[0277] As described above, the tree analysis unit 1030 is configured to generate occupancy codes while recursively dividing nodes until a predetermined size is reached.

[0278] In this embodiment, a method called "Octree" can be used, which recursively performs octree division on the above-mentioned rectangular parallelepiped, always treating it as a cube, and a method called "QtBt" can be used, which performs quadtree division and binary tree division in addition to octree division.

[0279] Here, whether or not to use “QtBt” is transmitted to the point cloud decoding device 200 as control data.

[0280] Alternatively, predictive coding using an arbitrary tree structure may be specified. In this case, the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoding device 200 as control data.

[0281] For example, the tree-structured control data may be configured so as to be decoded in accordance with the procedures described with reference to FIGS.

[0282] The approximate surface analyzer 1040 is configured to generate approximate surface information using the tree information generated by the tree analyzer 1030 .

[0283] Approximate surface information is used when, for example, decoding three-dimensional point cloud data of an object, in cases where the point cloud is densely distributed on the object surface, to approximate the area in which the point cloud exists using a small plane, rather than decoding each individual point cloud.

[0284] Specifically, the approximate surface analysis unit 1040 may be configured to generate approximate surface information using, for example, a method called "Trisoup." In addition, when decoding a sparse point cloud acquired by Lidar or the like, this process can be omitted.

[0285] The geometric information encoding unit 1050 is configured to generate a bit stream (geometric information bit stream) by encoding syntax such as the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040. Here, the bit stream may include, for example, the syntax described in FIG. 4.

[0286] The encoding process is, for example, a context-adaptive binary arithmetic encoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0287] The geometric information reconstruction unit 1060 is configured to reconstruct geometric information (the coordinate system assumed by the encoding process, i.e., the position information after coordinate transformation in the coordinate transformation unit 1010) of each point of the point cloud data to be encoded, based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040.

[0288] The frame buffer 1140 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 1060 as an input and store it as a reference frame.

[0289] When inter prediction is performed in the tree analysis unit 1030, the stored reference frame is read out from the frame buffer 1140 and used as a reference frame.

[0290] The color conversion unit 1070 is configured to perform color conversion when the input attribute information is color information. The color conversion does not necessarily have to be performed, and the presence or absence of the color conversion process is coded as part of the control data and transmitted to the point cloud decoding device 200.

[0291] The attribute transfer unit 1080 is configured to correct the attribute values ​​so as to minimize distortion of the attribute information, based on the position information of the input point cloud, the position information of the point cloud after reconstruction in the geometric information reconstruction unit 1060, and the attribute information after color change in the color conversion unit 1070. As a specific correction method, for example, the method described in Non-Patent Document 2 can be applied.

[0292] The RAHT unit 1090 is configured to receive the attribute information transferred by the attribute transfer unit 1080 and the geometric information generated by the geometric information reconstruction unit 1060 as input, and to generate residual information for each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform). As a specific process of RAHT, for example, the method described in the above-mentioned Non-Patent Document 2 can be used.

[0293] The LoD calculation unit 1100 is configured to receive the geometric information generated by the geometric information reconstruction unit 1060 as an input and generate a Level of Detail (LoD).

[0294] LoD is information for defining a reference relationship (a referencing point and a referenced point) to realize predictive coding, such as predicting attribute information of a certain point from attribute information of another point and encoding or decoding the prediction residual.

[0295] In other words, LoD is information that defines a hierarchical structure in which each point contained in geometric information is classified into multiple levels, and attributes of points belonging to lower levels are encoded or decoded using attribute information of points belonging to higher levels.

[0296] As a specific method for determining the LoD, for example, the method described in Non-Patent Document 2 mentioned above may be used.

[0297] The lifting unit 1110 is configured to generate residual information by a lifting process using the LoD generated by the LoD calculation unit 1100 and the attribute information after attribute transfer by the attribute transfer unit 1080.

[0298] As a specific example of the lifting process, the method described in the above-mentioned non-patent document 2 may be used.

[0299] The attribute information quantization unit 1120 is configured to quantize the residual information output from the RAHT unit 1090 or the lifting unit 1110. Here, a quantization step size of 1 is equivalent to no quantization being performed.

[0300] The attribute information encoding unit 1130 is configured to perform encoding processing using the quantized residual information and the like output from the attribute information quantization unit 1120 as syntax, and to generate a bit stream related to the attribute information (attribute information bit stream).

[0301] The encoding process is, for example, a context-adaptive binary arithmetic encoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0302] Through the above processing, the point group encoding device 100 is configured to perform encoding processing using position information and attribute information of each point in a point group as input, and to output a geometric information bit stream and an attribute information bit stream.

[0303] Furthermore, the above-mentioned point group encoding device 100 and point group decoding device 200 may be realized as a program that causes a computer to execute each function (each process).

[0304] In each of the above embodiments, the present invention has been described using the point cloud encoding device 100 and the point cloud decoding device 200 as examples, but the present invention is not limited to such examples and can be similarly applied to a point cloud encoding / decoding system having the functions of the point cloud encoding device 100 and the point cloud decoding device 200. [Industrial Applicability]

[0305] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0306] 10...Point cloud processing system 100...Point cloud encoding device 1010... Coordinate conversion section 1020...Geometric information quantization section 1030…Tree analysis section 1040…Approximate surface analysis section 1050...Geometric information encoding unit 1060...Geometric information reconstruction unit 1070…Color conversion section 1080…Attribute transfer section 1090…RAHT Section 1100…LoD calculation section 1110…Lifting section 1120...Attribute information quantization section 1130...Attribute information encoding unit 1140...Frame buffer 200...Point cloud decoding device 2010…Geometric Information Decoding Department 2020…Tree synthesis section 2030…Approximate surface synthesis part 2040...Geometric information reconstruction unit 2050…Inverse coordinate conversion section 2060…Attribute information decoding unit 2070...Inverse quantization section 2080…RAHT Division 2090…LoD calculation section 2100…Reverse lifting section 2110…Color inversion unit 2120...Frame buffer

Claims

1. A point cloud decoding device, comprising: Equipped with a tree synthesis unit, The tree synthesis unit includes: In the angular mode of predictive coding, the processing to be performed is switched depending on the sensor type or operation mode. A point cloud decoding device characterized in that, when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR data or the operation mode indicates the enablement of elevation angle component prediction and residual decoding, intra-prediction is performed for the coordinate values ​​r, θ, and φ of the node to be processed after polar coordinate transformation based on the values ​​of the parent node and ancestor node of the node to be processed.

2. The point cloud decoding device according to claim 1, characterized in that the tree synthesis unit performs the intra prediction using a value obtained by adding a difference between the value of the parent node and the value of the ancestor node to the value of the parent node as a predicted value.

3. The point cloud decoding device according to claim 1, characterized in that the tree synthesis unit performs the intra prediction using a value obtained by adding an average value of the differences between nodes in the most recently processed branch to the value of the parent node as a predicted value.

4. The point cloud decoding device according to claim 1, characterized in that the tree synthesis unit performs the intra prediction using a predicted value obtained by adding the value of the parent node to the difference between the node that is most similar to the value of the parent node in the branch processed immediately before and the child node of the node.

5. The point cloud decoding device according to any one of claims 1 to 4, characterized in that the tree synthesis unit has a plurality of intra prediction methods for the coordinate values ​​r, θ, and φ after polar coordinate transformation of the node to be processed, and switches the intra prediction method based on a flag.

6. 1. A point cloud decoding method, comprising: A step of switching a process to be performed depending on a sensor type or an operation mode in an angular mode of predictive coding; A point cloud decoding method characterized by comprising a step of performing intra-prediction for the coordinate values ​​r, θ, and φ of the node to be processed after polar coordinate transformation based on the values ​​of the parent node and ancestor node of the node to be processed when the mode is angular and the sensor type indicates Non-Spinning LiDAR data or the operation mode indicates the enablement of elevation angle component prediction and residual decoding.

7. A program for causing a computer to function as a point group decoding device, The point group decoding device comprises: Equipped with a tree synthesis unit, The tree synthesis unit includes: In the angular mode of predictive coding, the processing to be performed is switched depending on the sensor type or operation mode. A program characterized by performing intra-prediction for the coordinate values ​​r, θ, and φ of the node to be processed after polar coordinate transformation based on the values ​​of the parent node and ancestor node of the node to be processed when the angular mode is selected and the sensor type indicates Non-Spinning LiDAR data or the operation mode indicates the enablement of elevation angle component prediction and residual decoding.

Images