Point cloud decoding device, point cloud decoding method, and program

Abstract

To reduce an amount of decoding processing of attribute information.SOLUTION: A point cloud decoding device 200 of the present invention comprises an RAHT unit 2080 configured to search for adjacent nodes in a higher hierarchy in intra prediction and set a search range for adjacent nodes in the higher hierarchy on the basis of past search results.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a point cloud decoding apparatus, a point cloud decoding method, and a program.

Background Art

[0002] Conventionally, when searching for adjacent nodes in the intra prediction of RAHT, the maximum range in which adjacent nodes can exist is searched in the order of Mortion code.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the order of Mortion code, since the values of the Mortion codes of spatially adjacent nodes may be significantly different, there is a problem that the search range increases and the amount of processing related to the search increases.

[0005] Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide a point cloud decoding apparatus, a point cloud decoding method, and a program capable of reducing the amount of decoding processing of attribute information.

Means for Solving the Problems

[0006] A first feature of the present invention is a point cloud decoding apparatus, which includes a RAHT unit that searches for adjacent nodes in a higher layer in intra prediction and sets a search range for the adjacent nodes in the higher layer based on past search results.

[0007] A second feature of the present invention is a point cloud decoding method, which includes a step of searching for adjacent nodes in a higher layer in intra prediction and setting a search range for the adjacent nodes in the higher layer based on past search results.

[0008] A third feature of the present invention is a program for causing a computer to function as a point cloud decoding apparatus, which includes a RAHT unit that searches for adjacent nodes in a higher layer in intra prediction and sets a search range for the adjacent nodes in the higher layer based on past search results.

Advantages of the Invention

[0009] According to the present invention, it is possible to provide a point cloud decoding apparatus, a point cloud decoding method, and a program that can reduce the amount of decoding processing for attribute information.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0012] (First Embodiment) Hereinafter, with reference to FIGS. 1 to 24, a point cloud processing system 10 according to a first embodiment of the present invention will be described. FIG. 1 is a diagram showing the point cloud processing system 10 according to the present embodiment.

[0013] As shown in FIG. 1, the point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0014] The point cloud encoding device 100 is configured to generate encoded data (bitstream) by encoding an input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bitstream.

[0015] Note that the input point cloud signal and the output point cloud signal are composed of the position information and attribute information of each point in the point cloud. The attribute information is, for example, the color information or reflectivity of each point.

[0016] Here, such a bit stream may be transmitted from the point cloud encoding device 100 to the point cloud decoding device 200 via a transmission path. Further, the bit stream may be stored in a storage medium and then provided from the point cloud encoding device 100 to the point cloud decoding device 200.

[0017] (Point cloud decoding device 200) Hereinafter, with reference to FIG. 2, the point cloud decoding device 200 according to the present embodiment will be described. FIG. 2 is a diagram showing an example of the functional blocks of the point cloud decoding device 200 according to the present embodiment.

[0018] As shown in FIG. 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate conversion unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, a LoD calculation unit 2090, an inverse lifting unit 2100, an inverse color conversion unit 2110, and a frame buffer 2120.

[0019] The geometric information decoding unit 2010 is configured to input a bit stream related to geometric information (geometric information bit stream) among the bit streams output from the point cloud encoding device 100 and decode the syntax.

[0020] The decoding process is, for example, context-adaptive binary arithmetic decoding. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0021] The tree synthesis unit 2020 is configured to generate tree information indicating in which region of the decoding target space points exist, using the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating in which node of the tree described later the point cloud exists as inputs.

[0022] Note that the occupancy code decoding process may be configured to be performed inside the tree synthesis unit 2020.

[0023] This process can generate tree information by dividing the decoding target space into rectangular parallelepipeds, determining whether there are points in each rectangular parallelepiped by referring to the occupancy code, dividing the rectangular parallelepiped with points into multiple rectangular parallelepipeds, and recursively repeating the process of referring to the occupancy code.

[0024] Here, when decoding such an occupancy code, inter prediction described later may be used.

[0025] In this embodiment, a method called "Octree" that recursively performs octree division with the above rectangular parallelepiped always as a cube, and a method called "QtBt" that performs quadtree division and binary tree division in addition to octree division can be used. Whether to use "QtBt" is transmitted from the point cloud encoding device 100 side as control data.

[0026] Alternatively, when specified to use Predictive geometry coding by control data, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree configuration determined in the point cloud encoding device 100.

[0027] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree synthesis unit 2020 and decode the point cloud based on such approximate surface information.

[0028] The approximate surface information is, for example, when the point cloud is densely distributed on the object surface when decoding the three-dimensional point cloud data of the object, etc., instead of decoding each individual point cloud, it is an approximation of the existence region of the point cloud represented by small planes.

[0029] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information and decode the point cloud by, for example, a method called "Trisoup". A specific processing example of "Trisoup" will be described later. Also, when decoding a sparse point cloud obtained by Lidar or the like, this process can be omitted.

[0030] The geometric information reconstruction unit 2040 is configured to reconstruct the geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point cloud data to be decoded based on the tree information generated by the tree synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0031] The inverse coordinate conversion unit 2050 is configured to convert the geometric information reconstructed by the geometric information reconstruction unit 2040 from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal and output the position information.

[0032] The frame buffer 2120 is configured to store the geometric information reconstructed by the geometric information reconstruction unit 2040 as a reference frame. The stored reference frame is read from the frame buffer 2130 and used as a reference frame when performing inter-prediction of frames with different times in the tree synthesis unit 2020.

[0033] Here, which reference frame at which time is used for each frame may be determined based on, for example, control data transmitted as a bitstream from the point cloud encoding device 100.

[0034] The attribute information decoding unit 2060 is configured to input a bitstream related to attribute information (attribute information bitstream) among the bitstreams output from the point cloud encoding device 100 and decode the syntax.

[0035] The decoding process is, for example, context-adaptive binary arithmetic decoding. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0036] Also, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

[0037] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and a quantization parameter that is one of the control data decoded by the attribute information decoding unit 2060, and generate inverse quantized residual information.

[0038] The inverse quantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090 according to the characteristics of the point cloud to be decoded. Which one to output is specified by the control data decoded by the attribute information decoding unit 2060.

[0039] The RAHT unit 2080 takes as input the inverse quantized residual information generated by the inverse quantization unit 2070 and the geometric information generated by the geometric information reconstruction unit 2040, and is configured to decode the attribute information of each point using a type of Haar transform (in the decoding process, inverse Haar transform) called RAHT (Region Adaptive Hierarchical Transform). The information to be decoded is the direct current component (DC coefficient) and the alternating current component (AC coefficient) of the attribute information generated by using RAHT in the encoding process, and in the decoding process, it is converted into attribute information by using the inverse transform of RAHT. As a specific process of RAHT, for example, the method described in Non-Patent Document 1 can be used.

[0040] The LoD calculation unit 2090 takes as input the geometric information generated by the geometric information reconstruction unit 2040 and is configured to generate LoD (Level of Detail).

[0041] The LoD is information for defining a reference relationship (a point to be referenced and a point to be referred to) for realizing predictive coding such as predicting the attribute information of one point from the attribute information of another point and encoding or decoding the prediction residual.

[0042] In other words, the LoD is information that defines a hierarchical structure in which each point included in the geometric information is classified into a plurality of levels, and for points belonging to a lower level, the attributes are encoded or decoded using the attribute information of points belonging to a higher level.

[0043] As a specific method for determining the LoD, for example, the method described in Non-Patent Document 1 above may be used.

[0044] The inverse lifting unit 2100 is configured to decode the attribute information of each point based on the hierarchical structure defined by the LoD using the LoD generated by the LoD calculation unit 2090 and the inverse quantized residual information generated by the inverse quantization unit 2070. As a specific process of inverse lifting, for example, the method described in Non-Patent Document 1 above can be used.

[0045] The inverse color conversion unit 2110 is configured to perform an inverse color conversion process on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information to be decoded is color information and color conversion has been performed on the point cloud encoding device 100 side. Whether to execute such an inverse color conversion process is determined by the control data decoded by the attribute information decoding unit 2060.

[0046] The point cloud decoding device 200 is configured to decode and output the attribute information of each point in the point cloud by the above processing.

[0047] (Geometric information decoding unit 2010) Hereinafter, the control data decoded by the geometric information decoding unit 2010 will be described with reference to FIGS. 3 to 4.

[0048] FIG. 3 is an example of the configuration of the encoded data (bit stream) received by the geometric information decoding unit 2010.

[0049] First, the bit stream may include GPS2011. GPS2011 is also called a geometry parameter set and is a set of control data related to the decoding of geometric information. Specific examples will be described later. Each GPS2011 includes at least GPS id information for identifying each one when there are a plurality of GPS2011.

[0050] Second, the bit stream may include GSH2012A / 2012B. GSH2012A / 2012B is also called a geometry slice header or a geometry data unit header and is a set of control data corresponding to the slices described later. Hereinafter, the description will be made using the term "slice", but the slice can also be read as a data unit. Specific examples will be described later. GSH2012A / 2012B includes at least GPS id information for specifying the GPS2011 corresponding to each GSH2012A / 2012B.

[0051] Third, the bit stream may include slice data 2013A / 2013B after GSH2012A / 2012B. The slice data 2013A / 2013B includes data obtained by encoding geometric information.

[0052] As described above, the bit stream has a configuration in which one GSH2012A / 2012B and one GPS2011 correspond to each slice data 2013A / 2013B.

[0053] As described above, since the GPS id information is used in GSH2012A / 2012B to specify which GPS2011 to refer to, a common GPS2011 can be used for a plurality of slice data 2013A / 2013B.

[0054] In other words, GPS2011 does not necessarily need to be transmitted for each slice. For example, as shown in FIG. 3, immediately before GSH2012B and slice data 2013B, the bitstream can be configured such that GPS2011 is not encoded.

[0055] Note that the configuration of FIG. 3 is merely an example. As long as GSH2012A / 2012B and GPS2011 correspond to each slice data 2013A / 2013B, elements other than those described above may be added as components of the bitstream.

[0056] For example, as shown in FIG. 3, the bitstream may include a sequence parameter set (SPS) 2001. Similarly, at the time of transmission, it may be shaped into a configuration different from that of FIG. 3. Further, it may be transmitted as a single bitstream by being combined with the bitstream decoded by the attribute information decoder 2060 described later.

[0057] FIG. 4 is an example of the syntax configuration of GPS2011.

[0058] Note that the syntax names described below are merely examples. As long as the functions of the syntaxes described below are the same, the syntax names may be different.

[0059] GPS2011 may include GPS id information (gps_geom_parameter_set_id) for identifying each GPS2011.

[0060] Note that the Descriptor column in FIG. 4 indicates how each syntax is encoded. ue(v) means that it is an unsigned 0th order exponential Golomb code, and u(1) means that it is a 1-bit flag.

[0061] GPS2011 may include a flag (geom_tree_type) for controlling the tree type in the tree synthesis unit 2020.

[0062] For example, when the value of geom_tree_type is "1", it may be defined to use Predictive geometry coding, and when the value of geom_tree_type is "0", it may be defined to use Octree.

[0063] GPS2011 may include a flag (geom_angular_enabled) in the tree synthesis unit 2020 to control whether to perform processing in the Angular mode.

[0064] For example, when the value of geom_angular_enabled is "1", it may be defined to perform the processing of Predictive geometry coding in the Angular mode, and when the value of geom_angular_enabled is "0", it may be defined not to perform the processing of Predictive geometry coding in the Angular mode.

[0065] GPS2011 may include a flag (ptree_ang_azimuth_scaling_enabled) in the tree synthesis unit 2020 to control whether it is in the adaptive azimuth quantization mode in the Angular mode. The adaptive azimuth quantization mode is a mode that performs adaptive azimuth quantization according to the radius.

[0066] For example, when the value of ptree_ang_azimuth_scaling_enabled is "1", it may be defined to perform adaptive azimuth quantization according to the radius, and when the value of ptree_ang_azimuth_scaling_enabled is "0", it may be defined not to perform adaptive azimuth quantization according to the radius.

[0067] Also, it may be used as a flag to control whether to use the predictor list in the calculation (selection) of the predictor in the Angular mode.

[0068] For example, when the value of ptree_azimuth_scaling_enabled is "1", it may be defined that the predictor list is used in the calculation of such a predictor, and when the value of ptree_ang_azimuth_scaling_enabled is "0", it may be defined that the predictor list is not used in the calculation of such a predictor.

[0069] GPS2011 may include a value (ptree_ang_azimuth_step_minus1) related to the rotation speed of the laser used to calculate the predicted value of the azimuth angle in the Angular mode in the tree synthesis unit 2020.

[0070] (Tree synthesis unit 2020) Hereinafter, an example of the operation of the tree synthesis unit 2020 will be described with reference to FIGS. 16 to 17.

[0071] FIG. 16 is a flowchart showing an example of the processing in the tree synthesis unit 2020. Hereinafter, an example of the case of synthesizing a tree using "Predictive geometry coding" will be described.

[0072] Predictive geometry coding is also called Predictive Tree. Predictive geometry coding is a means of decoding the predicted position information based on an arbitrary tree structure determined on the side of the point cloud encoding device 100 and the residual of the position information of the point cloud data, and adding the two to decode the position information of the point cloud data.

[0073] As shown in FIG. 16, in step S1601, the tree synthesis unit 2020 determines whether the decoding of the position information of all the point cloud data included in the slice is completed.

[0074] This process can determine whether the processing of all points is completed by, for example, transmitting information indicating the number of point cloud data included in the slice to GSH and comparing this number of point cloud data with the number of data that has already been processed.

[0075] When the decoding of the position information of all point cloud data is completed, this operation proceeds to step S1613 and ends the process. When the decoding of the position information of all point cloud data is not completed, this operation proceeds to step S1602.

[0076] In step S1602, the tree synthesis unit 2020 sets the parent node of the node to be decoded (processing target node) of the point cloud data.

[0077] For example, the tree synthesis unit 2020 decodes the number of child nodes of each node to be decoded and stores the index of the node to be decoded by the number of such child nodes.

[0078] When the tree synthesis unit 2020 processes the node to be decoded after a certain node, it may refer to the array of indexes of the node, obtain one index stored at the end of the array, and set the node with the obtained index as the parent node of the node to be decoded.

[0079] After the setting of such a parent node is completed, this operation proceeds to step S1603.

[0080] In step S1603, the tree synthesis unit 2020 determines whether to perform processing in Angular mode.

[0081] For example, the tree synthesis unit 2020 can determine whether to perform processing in Angular mode by referring to the value of geom_angular_enabled described above.

[0082] When performing processing in Angular mode, this operation proceeds to step S1604. When not performing processing in Angular mode, this operation proceeds to step S1610.

[0083] In step S1604, the tree synthesis unit 2020 decodes the predictor information and the spherical coordinate residual to be used in step S1605. Here, the spherical coordinate residual indicates the residuals of the radius, azimuth angle, and laser ID. When such decoding is completed, this operation proceeds to step S1605.

[0084] In step S1605, the tree synthesis unit 2020 predicts the position information based on the predictor information decoded in step S1604. Here, the predictor information is the predictor index or prediction mode.

[0085] In such processing, the tree synthesis unit 2020 first determines the type of predictor to be used for prediction.

[0086] For example, the tree synthesis unit 2020 may determine whether to perform processing in the adaptive azimuth quantization mode based on the value of ptree_ang_azimuth_scaling_enabled, and determine the type of predictor to be used based on such determination result.

[0087] For example, in the case of the adaptive azimuth quantization mode, the tree synthesis unit 2020 may select the predictor to be used based on the decoded prediction mode from among a plurality of predictors calculated using a tree structure.

[0088] Alternatively, when performing processing in the adaptive azimuth quantization mode, the tree synthesis unit 2020 may hold the position information of the decoded nodes in a list as predictors, refer to the predictor assigned to the decoded predictor index from the list, and select it as the type of predictor to be used.

[0089] When the type of predictor is determined, the tree synthesis unit 2020 uses such predictor as the predicted value of the position information.

[0090] After the prediction of such position information is completed, this operation proceeds to step S1606.

[0091] In step S1606, the tree synthesis unit 2020 reconstructs the spherical coordinates. In such processing, the tree synthesis unit 2020 reconstructs the spherical coordinates by adding the decoded spherical coordinate residuals and the predictor.

[0092] After such reconstruction is completed, this operation proceeds to step S1607.

[0093] In step S1607, the tree synthesis unit 2020 reconstructs the orthogonal integer coordinates. In such processing, based on the reconstructed spherical coordinates, the tree synthesis unit 2020 can perform a conversion from the spherical coordinates to the orthogonal integer coordinates of the spherical coordinates. As a specific method, for example, it can be realized by the method described in Non-Patent Document 1.

[0094] After such reconstruction of the orthogonal integer coordinates is completed, this operation proceeds to step S1608.

[0095] In step S1608, the tree synthesis unit 2020 decodes the orthogonal integer coordinate residuals.

[0096] After the decoding of the orthogonal integer coordinate residuals is completed, this operation proceeds to step S1609.

[0097] In step S1609, the tree synthesis unit 2020 reconstructs the original coordinates. In such processing, the tree synthesis unit 2020 reconstructs the original coordinates by adding the decoded orthogonal integer coordinate residuals and the reconstructed orthogonal integer coordinates.

[0098] After such reconstruction of the original coordinates is completed, this operation returns to step S1601.

[0099] In step S1610, the tree synthesis unit 2020 predicts the position information. Specifically, the tree synthesis unit 2020 selects a predictor and uses such predictor as the predicted value of the position information.

[0100] For example, the tree synthesis unit 2020 may select a predictor from among a plurality of predictors calculated based on the tree structure according to the decoded predictor mode.

[0101] After the prediction of such position information is completed, this operation proceeds to step S1611.

[0102] In step S1611, the tree synthesis unit 2020 decodes the orthogonal integer coordinate residual.

[0103] After the decoding of the orthogonal integer coordinate residual is completed, this operation proceeds to step S1612.

[0104] In step S1612, the tree synthesis unit 2020 reconstructs the original coordinates. In such process, the tree synthesis unit 2020 reconstructs the original coordinates by adding the residual of the orthogonal integer coordinates decoded in step S1611 and the position information predicted in step S1610.

[0105] After the reconstruction of such original coordinates is completed, this operation returns to step S1601.

[0106] FIG. 17 is a flowchart showing an example of the decoding process of the predictor information and the spherical coordinate residual in step S1604.

[0107] As shown in FIG. 17, in step S1701, the tree synthesis unit 2020 determines whether it is in the adaptive azimuth quantization mode based on the value of ptree_ang_azimuth_scaling_enabled.

[0108] If it is in the adaptive azimuth quantization mode, this operation proceeds to step S1702. On the other hand, if it is not in the adaptive azimuth quantization mode, this operation proceeds to step S1703.

[0109] In step S1702, the tree synthesis unit 2020 decodes the predictor index. After the decoding of the predictor index is completed, this operation proceeds to step S1704.

[0110] In step S1703, the tree synthesis unit 2020 decodes the prediction mode. After the decoding of the prediction mode is completed, this operation proceeds to step S1704.

[0111] In step S1704, the tree synthesis unit 2020 decodes the number of azimuth angle steps. After the decoding of the number of azimuth angle steps is completed, this operation proceeds to step S1705.

[0112] In step S1705, the tree synthesis unit 2020 decodes the spherical coordinate residual. The tree synthesis unit 2020 may perform such decoding using the method described in Non-Patent Document 2. After the decoding is completed, this operation proceeds to step S1706 and the process ends.

[0113] (Attribute information decoding unit 2060) Hereinafter, the control data decoded by the attribute information decoding unit 2060 will be described with reference to FIGS. 5 to 6 and FIG. 24.

[0114] FIG. 5 is an example of the configuration of the encoded data (bit stream) received by the attribute information decoding unit 2060, and FIGS. 6 and 24 are examples of the syntax configuration of the APS2611 shown in FIG. 5.

[0115] Note that the syntax names described below are merely examples. As long as the functions of the syntaxes described below are the same, the syntax names may be different.

[0116] APS2611 may include APS id information (aps_geom_parameter_set_id) for identifying each APS2611.

[0117] Note that the Descriptor column in Fig. 6 indicates how each syntax is encoded. se(v) means it is a signed zero-order exponential Golomb code, ue(v) means it is an unsigned zero-order exponential Golomb code, and u(1) means it is a 1-bit flag.

[0118] APS2611 may include a flag (attr_coding_type) for controlling whether to output the inverse quantized residual information to either the RAHT unit 2080 or the LoD calculation unit 2090 in the inverse quantization unit 2070.

[0119] For example, when the value of attr_coding_type is "1", it may be defined to output to the LoD calculation unit 2090, and when the value of attr_coding_type is "0", it may be defined to output to the RAHT unit 2080.

[0120] APS2611 may include a flag (raht_prediction_enabled) for controlling whether to perform prediction of attribute information in the RAHT unit 2080.

[0121] For example, when the value of raht_prediction_enabled is "1", it may be defined to perform prediction of attribute information, and when the value of raht_prediction_enabled is "0", it may be defined not to perform prediction of attribute information.

[0122] APS2611 may include a flag (raht_subnode_prediction_enable_flag) for controlling whether to use subnodes for prediction of attribute information in the RAHT unit 2080.

[0123] For example, when the value of raht_subnode_prediction_enable_flag is "1", it may be defined that subnodes are used for predicting attribute information, and when the value of raht_subnode_prediction_enable_flag is "0", it may be defined that subnodes are not used for predicting attribute information.

[0124] APS2611 may include weight parameters (raht_prediction_weights) for intra prediction of attribute information in the RAHT unit 2080.

[0125] For example, the value of raht_prediction_weights may be defined according to the way of adjacency between the node to be decoded and the adjacent nodes used for intra prediction.

[0126] APS2611 may include a flag (raht_smoothing_enable_flag) for controlling whether to perform smoothing after intra prediction of attribute information in the RAHT unit 2080.

[0127] For example, when the value of raht_smoothing_enable_flag is "1", it may be defined that smoothing is performed after predicting attribute information, and when the value of raht_smoothing_enable_flag is "0", it may be defined that no smoothing is performed.

[0128] APS2611 may include weight parameters (raht_smoothing_weighted_average_weights) for performing smoothing by weighted average after intra prediction of attribute information in the RAHT unit 2080.

[0129] For example, up to 8 such weight parameters may be defined according to the way of adjacency between the node to be decoded and each subnode of the same parent node of the node to be decoded.

[0130] APS2611 may include a weight parameter (raht_smoothing_clipping_weights) for performing smoothing by clipping after performing intra prediction of attribute information in the RAHT unit 2080.

[0131] For example, up to eight such weight parameters may be defined according to the way of adjacency between the node to be decoded and each sub-node of the same parent node of the node to be decoded.

[0132] APS2611 may include a threshold value (raht_smoothing_clipping_threshold) for performing smoothing by clipping after performing intra prediction of attribute information in the RAHT unit 2080.

[0133] APS2611 may include a flag (raht_inter_prediction_enabled) for controlling whether to perform inter prediction of attribute information in the RAHT unit 2080.

[0134] For example, when the value of raht_inter_prediction_enabled is "1", it may be defined that the prediction of attribute information is performed, and when the value of raht_inter_prediction_enabled is "0", it may be defined that the prediction of attribute information is not performed.

[0135] APS2611 may include a value (raht_inter_prediction_depth_minus1) indicating the layer for enabling inter prediction of attribute information in the RAHT unit 2080.

[0136] For example, when raht_inter_prediction_depth_minus1 is "N - 1", inter prediction may be enabled up to the upper N layers of the Octree structure.

[0137] When APS2611 predicts attribute information (for example, when the value of raht_prediction_enabled is "1"), as shown in FIG. 24, it may additionally include the following syntax.

[0138] APS2611 may include syntax (raht_prediction_search_range) indicating the maximum search range during the search for the upper - layer nodes of RAHT.

[0139] APS2611 may include a flag (raht_last_comp_pred_enabled) indicating whether to predict the Cr signal from the Cb signal. For example, when the value of such a flag is "1", it may be defined that the Cr signal is predicted from the Cb signal, and when the value of such a flag is "0", the Cr signal is not predicted from the Cb signal.

[0140] When APS2611 predicts the Cr signal from the Cb signal, it may additionally include syntax (raht_last_comp_pred_coeff_diff[dpth]) indicating the prediction coefficient value.

[0141] The attribute information decoding unit 2060 may decode such prediction coefficients as different values for each layer of RAHT. Also, the attribute information decoding unit 2060 may decode the value of such a prediction coefficient not as it is, but as a difference value from the coefficient value of the upper layer (already decoded).

[0142] APS2611 may include a flag (raht_inter_comp_pred_enabled) indicating whether to predict the color - difference signals (Cb signal and Cr signal) from the luminance signal. For example, when the value of such a flag is "1", it may be defined that the color - difference signals (Cb signal and Cr signal) are predicted from the luminance signal, and when the value of such a flag is "0", the color - difference signals (Cb signal and Cr signal) are not predicted from the luminance signal.

[0143] When predicting chrominance signals (Cb signal and Cr signal) from the luminance signal, APS2611 may additionally include a syntax (raht_inter_comp_pred_coeff_diff[dpth]) indicating prediction coefficient values.

[0144] The attribute information decoding unit 2060 may decode different values for such prediction coefficients for each layer of RAHT. Also, the attribute information decoding unit 2060 may decode different values for such prediction coefficients for each component to be predicted (Cb signal and Cr signal). Further, the attribute information decoding unit 2060 may decode the values of such prediction coefficients not as they are, but as difference values from the coefficient values of the upper layer (already decoded).

[0145] When predicting the Cr signal from the Cb signal, or when predicting chrominance signals (Cb signal and Cr signal) from the luminance signal, APS2611 may additionally include a syntax (raht_coeff_calc_range) indicating the reference range when calculating prediction coefficients on the side of the point group decoding device 200. (RAHT unit 2080) An example of the processing of the RAHT unit 2080 will be described with reference to FIGS. 7 to 15.

[0146] FIG. 7 is a flowchart showing an example of the processing of the RAHT unit 2080.

[0147] As shown in FIG. 7, in step S28001, the RAHT unit 2080 recursively divides a node into an octree until it reaches a predetermined size using a method called Octree. After such division is completed, this operation proceeds to step S28002.

[0148] In step S28002, the RAHT unit 2080 totals the number of points belonging to the lower layer of the node in each node divided by the Octree.

[0149] Specifically, the RAHT unit 2080 sequentially scans the nodes of a certain hierarchy and records the number of points belonging to each node. Next, the RAHT unit 2080 sums up the number of points recorded in the child nodes of each node in the nodes of the upper hierarchy and calculates the number of points belonging to each node.

[0150] The RAHT unit 2080 repeats the above scanning in order from the lowest layer to the topmost layer. The total number of acquired points is used as the weight for the inverse transform of RAHT in step S28005 described later. After such calculation is completed, this operation proceeds to step S28003.

[0151] In step S28003, the RAHT unit 2080 decodes the DC coefficient of the node belonging to the topmost layer of the Octree. Alternatively, the RAHT unit 2080 may calculate the DC coefficient by predicting the DC coefficient using intra prediction, decoding the prediction residual of the DC coefficient, and adding them together.

[0152] After the decoding of such DC coefficient is completed, the RAHT unit 2080 calculates the attribute value Aroot of the root node using the total number of points wroot belonging to the root node acquired in step S28002 and the decoded DC coefficient DCroot by the following formula.

[0153]

Equation

[0154] In step S28004, the RAHT unit 2080 determines whether the decoding of the attribute information of all the nodes included in the layer is completed.

[0155] If not completed, this operation proceeds to step S28005, and if completed, this operation proceeds to step S28007.

[0156] In step S28005, the RAHT unit 2080 decrypts the AC coefficients. Specifically, this will be described later. After such decryption is completed, this operation proceeds to step S28006.

[0157] In step S28006, the RAHT unit 2080 calculates the attribute value using the inverse transformation of RAHT based on the total number of points belonging to the lower hierarchy of each aggregated node, the decrypted AC coefficients, and the DC coefficients calculated from the nodes in the upper hierarchy by the method described later.

[0158] Here, the inverse transformation of RAHT is performed in units of 8 nodes of 2×2×2 that are octree-divided.

[0159] Specifically, the attribute values A1, A2, … A k are obtained by the following formula (1) using the DC coefficient DC of the node holding k sub-nodes, the AC coefficients AC1, AC2, … AC k-1 and the total number of points w = w1, w2, … w belonging to the lower hierarchy of each sub-node. k are obtained by the following formula (1).

[0160]

Equation

[0161] It is assumed that such conversion processing is repeatedly performed in the order from the nodes in the upper hierarchy to the nodes in the lower hierarchy.

[0162]

Equation

[0163] In step S28007, the RAHT unit 2080 determines whether the decoding of all levels of nodes is completed.

[0164] If not completed, this operation moves the processing target layer to the next lower layer and proceeds to step S28004. If completed, this operation proceeds to step S28008 and ends the processing.

[0165] FIG. 8 is a flowchart showing an example of the processing in step S28004.

[0166] As shown in FIG. 8, in step S28101, the RAHT unit 2080 determines whether to predict the AC coefficients. When making such a determination, the RAHT unit 2080 may refer to raht_prediction_enabled and use its value.

[0167] The RAHT unit 2080 may decode a flag indicating whether to perform prediction of the AC coefficients in the current processing target node and use the value of such a flag.

[0168] Such a flag may be decoded for each node or for each layer. Such a flag may be decoded only when the value of raht_prediction_enabled is the value "1" indicating that prediction is valid. Such a flag may be included in the slice data.

[0169] As a result of the determination, if the AC coefficients are not predicted, this operation proceeds to step S28102, and if the AC coefficients are predicted, this operation proceeds to steps S28103 and S28104.

[0170] In step S28102, the RAHT unit 2080 decodes the AC coefficients. After such decoding is completed, this operation proceeds to step S28106 and ends the processing.

[0171] In step S28103, the RAHT unit 2080 decodes the AC coefficient residuals.

[0172] For example, the RAHT unit 2080 may perform inverse quantization processing on the quantized AC coefficient residuals decoded from the bit stream based on the quantization parameters decoded from the bit stream, and calculate the inverse quantized AC coefficient residuals. In such a case, the inverse quantized AC coefficient residuals correspond to the output (decoded AC coefficient residuals) in step S28103.

[0173] Alternatively, for example, the RAHT unit 2080 may use the quantized AC coefficient residuals decoded from the bit stream as the output (decoded AC coefficient residuals) in step S28103 as they are.

[0174] Also, the RAHT unit 2080 may decode the AC coefficient residuals by the method shown in FIG. 22.

[0175] FIG. 22 is a flowchart showing an example of the process of step S28103. Note that FIG. 22 is a flowchart assuming a case where the attribute signal to be decoded has a plurality of components (for example, a luminance signal (Y signal) and color difference signals (Cb signal and Cr signal), etc.).

[0176] As shown in FIG. 22, in step 2201, the RAHT unit 2080 determines whether or not the decoding of the AC coefficient residuals of all components in the AC coefficient is completed.

[0177] If the decoding of the AC coefficient residuals of all components is completed, this operation proceeds to step S2207 and the process ends. On the other hand, if the decoding of the AC coefficient residuals of all components is not completed, this operation proceeds to step S2202.

[0178] In step S2202, the RAHT unit 2080 decodes the quantized AC coefficient residuals of the component of the AC coefficient from the bit stream.

[0179] Note that, prior to the process of step S2202, the RAHT unit 2080 decodes the quantized AC coefficient residual from the bit stream and stores it in the memory, and in step S2202, the AC coefficient residual stored in the memory may be read out. After decoding the quantized AC coefficient residual as described above, this operation proceeds to step S2203.

[0180] In step S2203, the RAHT unit 2080 performs inverse quantization processing on the quantized AC coefficient residual decoded in step S2202 based on the quantization parameter decoded from the bit stream, and calculates the inverse quantized AC coefficient residual. After the above process is completed, this operation proceeds to step S2204.

[0181] In step S2204, the RAHT unit 2080 checks whether the component is the target of inter-component prediction.

[0182] Whether the RAHT unit 2080 determines whether the component is the target of inter-component prediction based on the values of the syntax elements included in headers such as SPS, APS, and ASH.

[0183] For example, the RAHT unit 2080 may predict the color difference signal from the luminance signal. In such a case, the targets of inter-component prediction are the color difference signals (Cb signal and Cr signal).

[0184] Also, the RAHT unit 2080 may perform inter-component prediction between the color difference signals. For example, the RAHT unit 2080 may predict the Cr signal from the Cb signal. In such a case, the target of inter-component prediction is the Cr signal.

[0185] Here, it is assumed and described that decoding is performed in the order of the Y signal, Cb signal, and Cr signal. When the RAHT unit 2080 performs inter-component prediction, only the signals for which decoding has been completed prior to the signal to be predicted can be used.

[0186] When the component is not a prediction target between components, this operation returns to step S2201 and processes the next component. On the other hand, when the component is a prediction target between components, this operation proceeds to step S2205.

[0187] In step S2205, the RAHT unit 2080 performs inter-component prediction of the AC coefficient residual.

[0188] Hereinafter, the case of performing inter-component prediction of the Cb signal from the luminance signal will be described as an example, but inter-component prediction of the Cr signal from the luminance signal and the Cr signal from the Cb signal are also possible in the same manner.

[0189] Here, let the AC coefficient residual of the inverse-quantized luminance signal calculated in step S2203 be Y'. The predicted value Crp' of the AC coefficient residual of the Cb signal can be calculated, for example, as in the following equation.

[0190] Crp' = a × Y' + b Here, a and b are prediction coefficients.

[0191] The RAHT unit 2080 may decode the prediction coefficients from headers such as SPS, APS, and ASH.

[0192] Alternatively, the RAHT unit 2080 may calculate the prediction coefficients from other already decoded AC coefficient residuals. Specifically, for example, the RAHT unit 2080 may calculate and use the AC coefficient residual that minimizes the sum of squared errors when predicting the AC coefficient residual of the inverse-quantized Cb signal from the AC coefficient residual of the inverse-quantized luminance signal of other already decoded AC coefficient residuals using the above equation.

[0193] The above-described processing can be analytically calculated by using the least squares method. At this time, the RAHT unit 2080 may use all the already decoded AC coefficient residuals for calculating the prediction coefficients.

[0194] Further, the RAHT unit 2080 may use only the N most recently decoded AC coefficient residuals based on the AC coefficient residual as the prediction coefficient calculation. Here, the RAHT unit 2080 may determine the number of N prior to decoding, or may decode the value of N from a header such as SPS, APS, or ASH. For example, the RAHT unit 2080 may use the value of raht_coeff_calc_range as N.

[0195] Further, the RAHT unit 2080 may use a weighted least squares method instead of a simple least squares method. Specifically, the RAHT unit 2080 may perform the above weighting based on the absolute value of the difference between the inverse quantized AC coefficient residual value Y’ of the Y signal of the AC coefficient residual and the inverse quantized AC coefficient residual of the Y signal of the already decoded AC coefficient residual, or the squared value of such a difference. At this time, the RAHT unit 2080 may be defined such that the smaller such a difference is, the heavier the weight is (the larger the absolute value of the weighting factor is).

[0196] As described above, after obtaining the predicted value Crp’ of the AC coefficient residual, this operation proceeds to the next step S2206.

[0197] In step S2206, the RAHT unit 2080 updates the AC coefficient residual. Specifically, the RAHT unit 2080 uses the predicted value Crp’ to update the inverse quantized AC coefficient residual Cb’ of the Cb signal decoded in step S2203 as follows.

[0198] Cb’ = Cb’ + Crp’ After such an update is completed, this operation proceeds to step S2201 and processes the next component.

[0199] As described above, after such decoding is completed, this operation proceeds to step S28105.

[0200] In step S28104, the RAHT unit 2080 predicts the AC coefficients. For predicting the AC coefficients, Inter prediction may be used, or Intra prediction may be used.

[0201] The RAHT unit 2080 may first predict the attribute value, and then calculate the predicted value of the AC coefficients by RAHT. Specifically, it will be described later. After the prediction of such AC coefficients is completed, this operation proceeds to step S28105.

[0202] In step S28105, the RAHT unit 2080 adds the residual of the decoded AC coefficients and the predicted AC coefficients to reconstruct the AC coefficients. After such reconstruction is completed, this operation proceeds to step S28106 and ends the process.

[0203] FIG. 9 is a flowchart showing an example of the process of step S28104.

[0204] As shown in FIG. 9, in step S28107, the RAHT unit 2080 determines whether Inter prediction is valid. For the determination, the RAHT unit 2080 may refer to raht_inter_prediction_enabled and use its value. As a result of the determination, if Inter prediction is valid, this operation proceeds to step S28109, and if Inter prediction is invalid, this operation proceeds to step S28112.

[0205] In step S28109, the RAHT unit 2080 determines whether the depth of the layer containing the processing target node is less than or equal to a threshold value. For the threshold value, the RAHT unit 2080 may refer to raht_inter_prediction_depth_minus1 and use its value.

[0206] As a result of the determination, if the depth is less than or equal to the threshold value, this operation proceeds to step S28110, and if the depth is greater than the threshold value, this operation proceeds to step S28112.

[0207] In step S28110, the RAHT unit 2080 determines whether to perform inter prediction on the AC coefficient of the node to be processed.

[0208] For the determination, the RAHT unit 2080 may check whether inter prediction is feasible. If it is feasible, perform inter prediction; if not, it may not perform inter prediction even if not. Specifically, it will be described later.

[0209] For the determination, the RAHT unit 2080 may decode a flag indicating whether to perform inter prediction on the AC coefficient of the node to be processed and use the value of such a flag. Such a flag may be decoded for each node or for each layer. Such a flag may be decoded only when it is determined that inter prediction is feasible and the determination is made. Such a flag may be included in the slice data.

[0210] In step S28111, the RAHT unit 2080 performs inter prediction on the AC coefficient of the node to be processed. Specifically, it will be described later.

[0211] In step S28112, the RAHT unit 2080 performs intra prediction on the AC coefficient of the node to be processed. Specifically, it will be described later.

[0212] In step S28113, the process of step S28104 is terminated. Note that the conditional branch in step S28109 may be omitted.

[0213] In the process of inter prediction in step S28111, the same process as the intra prediction in step S28112 may be performed together, and the results of inter prediction and intra prediction may be combined for prediction. Specifically, it will be described later.

[0214] FIG. 10 is a flowchart showing an example of the process of intra prediction in step S28112.

[0215] As shown in FIG. 10, in step S28201, the RAHT unit 2080 determines whether to perform intra prediction using adjacent nodes in the sub-node layer. The RAHT unit 2080 may refer to the raht_subnode_prediction_enable_flag and use its value for the determination.

[0216] If the RAHT unit 2080 does not use the adjacent nodes in the sub-node layer, it performs intra prediction using only the adjacent nodes in the upper layer.

[0217] Here, the adjacent nodes in the upper layer are a total of 19 nodes including 6 nodes adjacent to the face, 12 nodes adjacent to the edge, and the parent node itself among the nodes adjacent to the parent node of the node to be decoded. Among them, there are 3 nodes adjacent to the face, 3 nodes adjacent to the edge, and 7 nodes of the parent node itself to which the node to be decoded is adjacent.

[0218] FIG. 11 is a diagram showing the relationship between the node to be decoded and the adjacent nodes in the upper layer.

[0219] If the RAHT unit 2080 uses the adjacent nodes in the sub-node layer, it performs intra prediction using the adjacent nodes in the upper layer and the adjacent nodes in the sub-node layer.

[0220] Here, the adjacent nodes in the sub-node layer are the decoded nodes among the sub-nodes of the adjacent nodes in the upper layer to which the node to be decoded is adjacent to the face or the edge.

[0221] FIG. 12 is a diagram showing the relationship between the node to be decoded and the adjacent nodes in the sub-node layer.

[0222] As a result of the determination, if intra prediction is performed without using the adjacent nodes in the sub-node layer, this operation proceeds to step S28202, and if intra prediction is performed using the adjacent nodes in the sub-node layer, this operation proceeds to step S28204.

[0223] In step S28202, the RAHT unit 2080 acquires the attribute values of the adjacent nodes in the upper layer.

[0224] The acquisition process of the attribute values can be realized, for example, in two steps: the search process for adjacent nodes and the process of acquiring the attribute values from the adjacent nodes identified by such a search process. Hereinafter, an example of the search process for adjacent nodes will be described with reference to FIGS. 19 to 21.

[0225] FIG. 19 is a flowchart showing an example of the search process for adjacent nodes in the upper layer of the node to be decoded.

[0226] In FIG. 19, similar to FIG. 11, the upper layer node of the node to be decoded is specially referred to as the parent node.

[0227] Further, hereinafter, in FIGS. 19 to 21, it is assumed that the Morton codes corresponding to the positions of the nodes in the upper layer are stored in ascending order of the Morton codes on a one-dimensional array (hereinafter referred to as the upper layer node array).

[0228] Furthermore, on the above-mentioned one-dimensional array, it is assumed that the index (hereinafter referred to as the parent node index) in which the Morton code of the parent node is stored is known in advance.

[0229] As shown in FIG. 19, in step S1901, the RAHT unit 2080 checks whether the search for all adjacent nodes in the upper layer has been completed.

[0230] If the search for all adjacent nodes has been completed, this operation proceeds to step S1905 and the process ends. On the other hand, if the search for all adjacent nodes has not been completed, this operation proceeds to step S1902.

[0231] In step S1902, the RAHT unit 2080 calculates the Morton code corresponding to the position of the adjacent node to be searched.

[0232] Specifically, the RAHT unit 2080 may calculate the Morton code corresponding to the position of the adjacent node to be searched by, for example, first converting the Morton code corresponding to the position of the parent node into the values of the orthogonal coordinates (x, y, z), second calculating the coordinates (x', y', z') of the adjacent node in the orthogonal coordinate space, and third converting the orthogonal coordinates of the adjacent node back into the Morton code.

[0233] After the calculation of such a Morton code is completed, this operation proceeds to step S1903.

[0234] In step S1903, the RAHT unit 2080 sets the search range. Here, the search range is the range of how far from the parent node index to search in the upper layer node array.

[0235] When the Morton code of the adjacent node is smaller than the Morton code of the parent node, the start point and the end point of the search range can be set as follows.

[0236] Search range = min(parent node's Morton code - adjacent node's Morton code, maximum search range) Start point of the search range = max(parent node index - search range, 0) End point of the search range = parent node index - 1 Here, min(a, b) is a function that returns the smaller value of the two arguments a and b. Also, max(a, b) is a function that returns the larger value of the two arguments a and b. The maximum search range is the pre-determined maximum value of the search range.

[0237] The value of the maximum search range may be decoded from headers such as SPS, APS, and ASH. For example, the RAHT unit 2080 may use the value of raht_prediction_search_range as the value of such a maximum search range.

[0238] On the one hand, when the Morton code of the adjacent node is larger than that of the parent node, the start point and the end point of the search range can be set as follows.

[0239] Search range = min(adjacent node's Morton code - parent node's Morton code, maximum search range) Start point of the search range = parent node index + 1 End point of the search range = min(parent node's Morton code + search range, number of nodes in the upper layer - 1) After determining the start point and the end point of the search as described above, this operation proceeds to step S1906.

[0240] In step S1906, the RAHT unit 2080 determines whether the adjacent node to be searched is not stored or may be stored between the start point and the end point set in step S1903.

[0241] If the RAHT unit 2080 determines that it is not stored, it determines that the adjacent node to be searched does not exist, proceeds to step S1901, and searches for the next adjacent node.

[0242] On the other hand, if the RAHT unit 2080 determines that it may be stored, it proceeds to the next step S1904 and conducts a search.

[0243] Here, such determination can be executed, for example, as follows.

[0244] When the Morton code of the adjacent node is smaller than that of the parent node, the RAHT unit 2080 checks the Morton code of the node stored at the index corresponding to the start point of the search range.

[0245] If the Morton code of the node stored in the index corresponding to the start point of the search range is greater than the Morton code of the adjacent node, the RAHT unit 2080 determines that no adjacent node to be searched is stored between the start point and the end point of the search.

[0246] Otherwise (if the Morton code of the node stored in the index corresponding to the start point of the search range is less than or equal to the Morton code of the adjacent node), the RAHT unit 2080 determines that there may be an adjacent node to be searched stored between the start point and the end point of the search.

[0247] Here, if the index corresponding to the start point of the search range is 0, the RAHT unit 2080 may determine that there may be an adjacent node to be searched stored between the start point and the end point of the search.

[0248] Also, if "parent node index - index corresponding to the start point" is less than or equal to the maximum search range, the RAHT unit 2080 may determine that there may be an adjacent node to be searched stored between the start point and the end point of the search.

[0249] On the other hand, if the Morton code of the adjacent node is greater than the Morton code of the parent node, the RAHT unit 2080 checks the Morton code of the node stored in the index corresponding to the end point of the search range.

[0250] If the Morton code of the node stored in the index corresponding to the end point of the search range is less than the Morton code of the adjacent node, the RAHT unit 2080 determines that no adjacent node to be searched is stored between the start point and the end point of the search.

[0251] Otherwise (when the Morton code of the node stored in the index corresponding to the end point of the search range is greater than or equal to the Morton code of the adjacent node), the RAHT unit 2080 determines that there may be an adjacent node to be searched stored between the start point and the end point of the search.

[0252] Here, when the index corresponding to the end point of the search range is "the number of nodes in the upper layer - 1", the RAHT unit 2080 may determine that there may be an adjacent node to be searched stored between the start point and the end point of the search.

[0253] Also, when "the index corresponding to the end point - the index of the parent node" is less than or equal to the maximum search range, the RAHT unit 2080 may determine that there may be an adjacent node to be searched stored between the start point and the end point of the search.

[0254] In other words, the above processing can be said to be a process of determining whether there is no or there may be an adjacent node to be searched within the search range based on the Morton code of the node stored at the start point or the end point of the search range.

[0255] In this way, prior to the search process, it is determined whether there may be or there is no node to be searched within the search range. If not, the search process can be omitted, thereby reducing unnecessary processing. When implemented in software, the execution time can be reduced, and when implemented in hardware, the power consumption can be reduced.

[0256] In step S1904, the RAHT unit 2080 searches for an element in the range of the upper layer node array specified by the above-mentioned start point and end point of the search that stores the same Morton code as the adjacent node.

[0257] Here, when an element storing the same Morton code as the adjacent node is found, the RAHT unit 2080 returns the index of that element.

[0258] On the other hand, when the RAHT unit 2080 does not find an element storing the same Morton code as the adjacent node, it returns a value (e.g., -1) indicating that the adjacent node could not be found. At this time, since the Morton codes are stored in ascending order in the upper-level node array, the RAHT unit 2080 can perform the search with fewer search times than a full search by using binary search or the like.

[0259] After the above processing is completed, this operation proceeds to step S1901 to search for the next adjacent node.

[0260] FIG. 20 is a flowchart showing an example of the upper-level adjacent node search process for the node to be decoded. Hereinafter, an example of the upper-level adjacent node search process will be described with reference to FIG. 20.

[0261] As shown in FIG. 20, in step 2001, the RAHT unit 2080 calculates the Morton codes of all adjacent nodes.

[0262] Here, the method for calculating the Morton code is the same as the method described in step S1902. The difference from step S1902 is that the Morton codes of all adjacent nodes (e.g., 19 nodes) are calculated. After such calculation processing is completed, this operation proceeds to step S2002.

[0263] In step S2002, the RAHT unit 2080 sorts the adjacent nodes in ascending order, for example, based on the Morton codes calculated in step S2001. Hereinafter, the search process for adjacent nodes is performed in ascending order of the Morton code values.

[0264] In step S2003, the RAHT unit 2080 determines whether the search for adjacent nodes with Morton codes smaller than that of the parent node among the adjacent nodes has been completed.

[0265] When such search is completed, this operation proceeds to step S2006. On the other hand, when such search is not completed, this operation proceeds to step S2004.

[0266] In step S2004, the RAHT unit 2080 sets the search range. For example, the RAHT unit 2080 sets such search range in the following manner.

[0267] Search range = min(min(parent node index - index of the other adjacent node discovered immediately before, Morton code of the parent node - Morton code of the adjacent node), maximum search range) Start point of the search range = max(parent node index - search range, 0) End point of the search range = parent node index - 1 Here, the difference from step S1903 is that the search range is set using the index of the other adjacent node discovered immediately before. Since adjacent nodes are searched in ascending order of the Morton code, it is guaranteed that the Morton code of the currently searched adjacent node is a larger value than the Morton code of the other adjacent node discovered immediately before. Similarly, it is guaranteed that the index of the element in the above hierarchical node array where the Morton code of the currently searched adjacent node is stored is larger than the index of the element where the Morton code of the other adjacent node discovered is stored. Therefore, by using the index of the other adjacent node discovered immediately before, the search range can be reduced, and the number of search processes and the time related to the search can be reduced.

[0268] Note that the RAHT unit 2080 may initialize the value of the index of the other adjacent node discovered immediately before to 0 and update it each time an adjacent node is discovered thereafter. After the above processing is completed, this operation proceeds to step S2005.

[0269] In step S2005, the RAHT unit 2080 performs the same search processing as in step S1904.

[0270] In step S2006, the RAHT unit 2080 changes the search order. Specifically, in the previous processing, the RAHT unit 2080 executed the search in ascending order of the Morton code of adjacent nodes, but this is changed to descending order of the Morton code. After changing such a search order, this operation proceeds to step S2007.

[0271] In step S2007, the RAHT unit 2080 determines whether the search for nodes whose Morton code is larger than that of the parent node among adjacent nodes has all been completed.

[0272] If all such searches have been completed, this operation proceeds to step S2009 and the process ends. If all such searches have not been completed, this operation proceeds to step S2008.

[0273] In step S2008, the RAHT unit 2080 sets the search range as follows. Here, the RAHT unit 2080 uses the index of the other adjacent node discovered immediately before, similar to step S2004.

[0274] Search range = min(min(Index of the other adjacent node discovered immediately before - Index of the parent node, Morton code of the adjacent node - Morton code of the parent node), Maximum search range) Start point of the search range = Index of the parent node + 1 End point of the search range = min(Morton code of the parent node + Search range, Number of nodes in the upper layer) Here, since the search is performed in descending order of the Morton code of adjacent nodes, it is guaranteed that the current Morton code being searched is smaller than the Morton code of the other adjacent node discovered immediately before.

[0275] Note that the index of the other adjacent node discovered immediately before is initialized with the number of nodes in the upper layer (= number of elements in the upper layer node array) at the timing of step S2006, and may be updated with such an index value each time an adjacent node is discovered thereafter.

[0276] As described above, after such a search range is set, this operation proceeds to step S2009.

[0277] In step S2009, the RAHT unit 2080 performs the same search process as in step S2005.

[0278] FIG. 21 is a flowchart showing an example of the adjacent node search process for the upper layer of the node to be decoded. Hereinafter, an example of the adjacent node search process for the upper layer will be described with reference to FIG. 21. For the same processes as in FIG. 20, the same reference numerals as in FIG. 20 are given and the description is omitted.

[0279] As shown in FIG. 21, in step S2101, the RAHT unit 2080 restores the search result recorded in step S2102 described later.

[0280] Specifically, the RAHT unit 2080 specifies the index value of the adjacent node by referring to the information stored in an array or the like.

[0281] In step S2102, when an adjacent node having a Morton code larger than that of the parent node is discovered, the RAHT unit 2080 records the index of the parent node as the adjacent node when the adjacent node becomes the parent node.

[0282] The adjacent node to the parent node is the parent node when viewed from the adjacent node (even when the adjacent node becomes the parent node). That is, there is a symmetric relationship.

[0283] In this embodiment, the RAHT unit 2080 processes the parent nodes in ascending order of the Morton code.

[0284] Therefore, the RAHT unit 2080 can reduce the search process by storing the index of the parent node as an adjacent node for adjacent nodes of the parent node whose Morton code is larger than that of the parent node (in case the adjacent node becomes the parent node).

[0285] That is, as described in step S2101, for adjacent nodes whose Morton code is smaller than that of the parent node, since they have been searched when the adjacent node was the parent node, the RAHT unit 2080 can omit re - executing the search process by storing the result.

[0286] As a result, the search process can be reduced by approximately half compared to the case where such storage is not performed.

[0287] As described above, after obtaining the attribute values of the adjacent nodes in the upper - level hierarchy, this operation proceeds to step S28203.

[0288] In step S28203, the RAHT unit 2080 predicts the attribute value of the node to be decoded.

[0289] The RAHT unit 2080 may predict the attribute value attr using the following formula, using the obtained attribute values attr of k upper - level adjacent nodes i and the weight w according to the type of adjacent node i i Here, as the weight w

[0290]

Equation

[0291] After the prediction of such attribute values is completed, this operation proceeds to step S28207.

[0292] In step S28204, the RAHT unit 2080 acquires the attribute values of the adjacent nodes in the upper layer.

[0293] Here, the target for acquiring the attribute values is a node among the adjacent nodes in the upper layer where the sub-nodes of each adjacent node are not decoded, or a node among the adjacent nodes in the upper layer where, even if the sub-nodes are decoded, there are no sub-nodes adjacent to the decoding target node by a face or an edge.

[0294] After the acquisition of such attribute values is completed, this operation proceeds to step S28205.

[0295] In step S28205, the RAHT unit 2080 acquires the attribute values of the adjacent nodes in the sub-node layer. After acquiring the attribute values of the adjacent nodes in the sub-node layer, this operation proceeds to step S28206.

[0296] In step S28206, the RAHT unit 2080 predicts the attribute value of the decoding target node.

[0297] The RAHT unit 2080 may predict the attribute value attr using the acquired attribute values attr of the k adjacent nodes in the upper layer and the adjacent nodes in the sub-node layer i and the weight w according to the type i of the adjacent node i by the following formula.

[0298]

Equation

[0299] After the prediction of such attribute values is completed, this operation proceeds to step S28207.

[0300] In step S28207, the RAHT unit 2080 converts the predicted attribute values into AC coefficients. The AC coefficients are generated by performing RAHT on the predicted attribute values. For example, the RAHT unit 2080 may use the method described in Non-Patent Document 1 as such a conversion method.

[0301] Note that in step S28207, instead of converting the predicted attribute values into AC coefficients, the RAHT unit 2080 may convert the residual of the decoded AC coefficients into the residual of the attribute values, and add the predicted value and the residual in the attribute value domain.

[0302] FIG. 23 is a flowchart showing an example of the intra prediction process of step S28112. Hereinafter, an example of the intra prediction process will be described with reference to FIG. 23. For the same processes as those in FIG. 10, the same reference numerals as those in FIG. 10 are given and the description thereof is omitted.

[0303] Also, in FIG. 23, the attribute information of the decoded point cloud data has a plurality of components, and the process in FIG. 23 will be described as the process for one of the components.

[0304] For each decoding target node, by applying the process in FIG. 23 to each component, the signals of all the components included in the decoding target node can be decoded.

[0305] In step S2201, the RAHT unit 2080 determines whether the component is a target for inter-component prediction.

[0306] Here, the RAHT unit 2080 may determine whether the component is a target for inter-component prediction based on the values of syntax elements included in headers such as SPS, APS, and ASH.

[0307] For example, when the value of raht_inter_comp_pred_enabled is "1", the RAHT unit 2080 may predict a chrominance signal from a luminance signal. In such a case, the targets for inter-component prediction are the chrominance signals (Cb signal and Cr signal).

[0308] Also, the RAHT unit 2080 may perform inter-component prediction between chrominance signals. For example, when the value of raht_last_comp_pred_enabled is "1", the RAHT unit 2080 may predict a Cr signal from a Cb signal. In such a case, the target for inter-component prediction is the Cr signal.

[0309] Here, it is assumed for explanation that decoding is performed in the order of the Y signal, Cb signal, and Cr signal. When performing inter-component prediction, only signals for which decoding has been completed earlier than the signal to be predicted can be used.

[0310] If the component is a target for inter-component prediction, this operation proceeds to step S2302; otherwise, this operation proceeds to step S28201.

[0311] In step S2302, the RAHT unit 2080 performs inter-component prediction.

[0312] In the following, the case of predicting the Cb signal from the luminance signal will be described as an example, but similarly, it is also possible to predict the Cr signal from the luminance signal and the Cr signal from the Cb signal. Here, let the AC coefficient of the luminance signal calculated in step S28207 be Y.

[0313] The predicted value Crp of the AC coefficient of the Cb signal can be calculated, for example, by the following formula.

[0314] Crp = a × Y + b Here, a and b are prediction coefficients. The RAHT unit 2080 may decode the prediction coefficients from headers such as SPS, APS, and ASH.

[0315] For example, the RAHT unit 2080 may decode such AC coefficients from the values of raht_last_comp_pred_coeff_diff[dpth] or raht_inter_comp_pred_coeff_diff[dpth].

[0316] Alternatively, the RAHT unit 2080 may calculate such prediction coefficients from other already decoded AC coefficient values.

[0317] Specifically, the RAHT unit 2080 may calculate and use, for example, the AC coefficient that minimizes the sum of squared errors when predicting the AC coefficient of the Cb signal by the above formula from the AC coefficient values of the luminance signal of other already decoded AC coefficients.

[0318] The above processing can be analytically calculated by using the least squares method. At this time, the RAHT unit 2080 may use all the already decoded AC coefficients for the calculation of such AC coefficients.

[0319] In addition, the RAHT unit 2080 may use only the N AC coefficients decoded most recently with respect to the said AC coefficient for the calculation of such AC coefficients. The RAHT unit 2080 may determine the number of N prior to decoding, or may decode the value of N from headers such as SPS, APS, and ASH.

[0320] For example, the RAHT unit 2080 may use the value of raht_coeff_calc_range as N.

[0321] Also, the RAHT unit 2080 may calculate the AC coefficient using a weighted least squares method instead of a simple least squares method.

[0322] Specifically, the RAHT unit 2080 may perform weighting based on the absolute value of the difference between the value Y of the Y signal of the AC coefficient and the Y signal of the already decoded AC coefficient, or the squared value of such a difference. At this time, it may be defined such that the smaller the difference, the heavier the weight.

[0323] In the above, the case where the RAHT unit 2080 performs the above prediction in the AC coefficient domain has been described as an example. However, the RAHT unit 2080 may perform the above prediction in the attribute signal domain.

[0324] That is, the RAHT unit 2080 may predict the attribute signal value of the Cb signal from the attribute signal value of the Y signal.

[0325] In the above, the example where the RAHT unit 2080 directly uses the attribute value predicted in step S28206 for the conversion of the AC coefficient in step S28207 has been described. However, the RAHT unit 2080 may perform the conversion of the AC coefficient after smoothing the predicted attribute value.

[0326] For example, as shown in FIG. 13, after predicting the attribute value, the RAHT unit 2080 may determine whether to perform smoothing in step S1301.

[0327] In such a determination, the RAHT unit 2080 may refer to raht_smoothing_enable_flag and use its value.

[0328] If smoothing is to be performed, this operation proceeds to step S1302. If smoothing is not to be performed, this operation proceeds to step S28207.

[0329] In step S1302, the RAHT unit 2080 may smooth the attribute value.

[0330] For example, for the smoothed attribute value Attrsmoothing of the node to be decoded, the RAHT unit 2080 may obtain it by calculating a weighted average as follows using the attribute value Attr predicted at the sub-node i in the same parent node as the node to be decoded i and the weight α i For the target sub-node i, the RAHT unit 2080 may use the node adjacent to the node to be decoded as the target sub-node i, or may use all sub-nodes in the same parent node.

[0331]

Equation

[0332] Also, the RAHT unit 2080 may use a hard-coded value as the weight α i or may refer to raht_smoothing_weighted_average_weights and use the value.

[0333] Also, for example, for the smoothed attribute value Attr smoothing of the node to be decoded, the predicted value Attr0 of the node to be decoded itself, the attribute value Attr predicted at the sub-node i other than the node to be decoded among the sub-nodes in the same parent node as the node to be decoded i , the weight β i and the threshold Th r For the target sub-node i, the RAHT unit 2080 may perform clipping as follows to obtain it using the predicted value Attr0 of the node to be decoded itself, the attribute value Attr predicted at the sub-node i other than the node to be decoded among the sub-nodes in the same parent node as the node to be decoded, the weight β, and the threshold Th.

[0334]

Equation

[0335] The function Clip3 that performs clipping is

[0336]

Number

[0337] Here, for the target sub-node i, the RAHT unit 2080 may use the nodes adjacent to the surface as the nodes to be decoded, or the nodes adjacent to the surface and the nodes adjacent to the edge, or all the sub-nodes in the same parent node.

[0338] Also, the RAHT unit 2080 may use the hard-coded value as the weight β i or refer to raht_smoothing_clipping_weights and use its value.

[0339] Also, the RAHT unit 2080 may use the hard-coded value as the threshold Th r or refer to raht_smoothing_clipping_threshold and use its value.

[0340] So far, an example where the RAHT unit 2080 decodes the AC coefficients of both the color difference signal and the luminance signal has been described. However, the RAHT unit 2080 may skip decoding the AC coefficients of the color difference signal only in the bottom layer of the Octree.

[0341] For example, as shown in FIG. 14, the RAHT unit 2080 may determine whether to skip decoding the AC coefficients of the color difference signal only in the bottom layer of the Octree in step S1401.

[0342] If skipping, this operation proceeds to step S1402. If not skipping, this operation proceeds to step S28004.

[0343] In step S1402, the RAHT unit 2080 determines whether the node to be decoded is the bottom layer of the Octree.

[0344] If it is the bottom layer, this operation proceeds to step S1403. If it is not the bottom layer, this operation proceeds to step S28004.

[0345] In step S1403, the RAHT unit 2080 decodes the AC coefficients other than the color difference signals.

[0346] For the decoding of the AC coefficients other than the color difference signals, the RAHT unit 2080 performs the same processing as in step S28004, sets the AC coefficients of the color difference signals to 0, and calculates the attribute values in the subsequent step S28005.

[0347] After the decoding of the AC coefficients other than the color difference signals is completed, this operation proceeds to step S28006.

[0348] FIG. 15 is a diagram showing an example of the inter prediction process in step S28111.

[0349] The RAHT unit 2080 predicts the AC coefficients of the node to be processed using the information of the reference node, which is the corresponding node in the reference frame within the reference frame. Here, the information of the reference node may be its attribute value or AC coefficient. Also, the reference frame may refer to another decoded frame, and its information may be included in the previous frame buffer 2120.

[0350] The RAHT unit 2080 may apply the same Octree structure as the frame to be processed to the reference frame. In such a case, there may be a possibility that nodes are set at positions without points. Such nodes are called empty nodes. If the reference node is an empty node, the RAHT unit 2080 may make the inter prediction unexecutable in step S28110.

[0351] The RAHT unit 2080 may apply an Octree independently of the frame to be processed with respect to the reference frame, and set an Octree structure different from that of the frame to be processed. In such a case, nodes may not necessarily exist at the same positions as those of the frame to be processed. If no reference node is found at the position corresponding to the node to be processed, the RAHT unit 2080 may make the inter prediction infeasible in step S28143.

[0352] If the reference node is an empty node, or if no reference node is found, the RAHT unit 2080 may estimate and interpolate the information of the reference node using the information of the nodes at neighboring positions within the reference frame.

[0353] For example, the RAHT unit 2080 may estimate and interpolate the average value of the attribute values or AC coefficients of the adjacent node, the nearest neighbor node, or the k-nearest neighbor nodes with respect to the reference node position as the attribute value or AC coefficient of the reference node, respectively.

[0354] The RAHT unit 2080 may predict, for example, the AC coefficient of the node to be processed from the attribute value of the reference node.

[0355] Specifically, the RAHT unit 2080 uses the value Attr inter of the decoded attribute value of the reference node to obtain the predicted value Attr pred of the attribute value of the node to be processed, and applies RAHT to the predicted value Attr pred of the attribute value of the node to be processed to obtain the predicted value AC pred of the AC coefficient of the node to be processed.

[0356] Attr pred = Attr inter AC pred = RAHT(Attr pred ) The RAHT unit 2080 may directly predict, for example, the AC coefficient of the node to be processed from the AC coefficient of the reference node.

[0357] Specifically, the RAHT unit 2080 calculates the value AC of the AC coefficient of the reference node using RAHT in the reference frame, and this value may be used as the predicted value AC of the AC coefficient of the node to be processed. inter Specifically, the RAHT unit 2080 calculates the value AC of the AC coefficient of the reference node using RAHT in the reference frame, and this value may be used as the predicted value AC of the AC coefficient of the node to be processed. pred Specifically, the RAHT unit 2080 calculates the value AC of the AC coefficient of the reference node using RAHT in the reference frame, and this value may be used as the predicted value AC of the AC coefficient of the node to be processed.

[0358] AC pred =AC inter The RAHT unit 2080 may record the AC coefficients of each node in the reference frame in the frame buffer 2120 for the AC coefficient of the reference node, and obtain the value by referring to the value in the frame buffer 2120. In such a case, when the AC coefficient of the reference node does not exist in the frame buffer 2120, the RAHT unit 2080 may make the inter prediction unexecutable in step S28110.

[0359] Note that the RAHT unit 2080 may multiply Attr inter and AC inter by α times respectively by the scaling factor α.

[0360] Attr pred =αAttr inter Or AC pred =αAC inter The coefficient α may take any real number. The coefficient α may be decoded for each node or for each layer. The coefficient α may be included in the slice data.

[0361] For example, the coefficient α may be defined using the depth of the layer as follows, and α' may be decoded instead of the coefficient α.

[0362] α = 1 + α'·2-depth For example, an integer β is defined as an integer taking a range from integer a to integer b, and β may be decoded. The coefficient α may be calculated as a value obtained by adding an integer c to the decoded β and then dividing by the integer c as follows.

[0363] α = (β + c) / c The integer β may be decoded using the exponential Golomb code.

[0364] Alternatively, the coefficient α may be derived by a decoder.

[0365] For example, it may be calculated as follows using the AC coefficient AC of the parent node of the node to be decoded parent and the inter-prediction value AC when decoding the parent node parent_inter as follows.

[0366] α = AC parent / AC parent_inter For example, the RAHT unit 2080 uses the AC coefficients AC of N adjacent nodes of the node to be decoded neighbor1 , AC neighbor2 , …, AC neighborN and the inter-prediction values AC when decoding each adjacent node neighbor_inter1 , AC neighbor_inter2 , …, AC neighbor_interN to calculate α so that the cost is minimized.

[0367] The cost may be, for example, the sum of the squared errors between the AC coefficient of each adjacent node and the AC coefficient predictor. The adjacent nodes may be, for example, only the nodes adjacent to the surface, or may be the nodes adjacent to the surface and the nodes adjacent to the edge.

[0368] The RAHT unit 2080 may perform a similar operation for the inter-prediction of the DC coefficient in step S28003.

[0369] DC pred = αDC inter Here, let the DC coefficient of the reference node be DC inter and the predicted value of the DC coefficient of the root node be DC pred as follows.

[0370] Further, the RAHT unit 2080 may calculate a predicted value of an attribute value or an AC coefficient by combining inter prediction and intra prediction.

[0371] For example, an example in the case where the RAHT unit 2080 obtains a prediction of an attribute value is shown below.

[0372] Attr pred =W inter ·Attr inter +W intra ·Attr intra Here, Attr inter and Attr intra are the inter prediction and intra prediction of the attribute value, respectively. Also, W inter and W intra are the weights of the inter prediction and intra prediction, respectively. W inter and W intra may be determined such that the intra prediction is more emphasized for deeper layers depending on the depth depth of the processing target layer. For example, W inter =1-depth / N W intra =depth / N N is the maximum value of the depth of the layer where the inter prediction is effective. The combination of the inter prediction and the intra prediction may be effective only at a specific layer. For example, the combination of the inter prediction and the intra prediction may be effective only when M < depth < N. M is an arbitrary real number less than N and may be decoded as header information such as APS.

[0373] (Point cloud encoding device 100) Hereinafter, with reference to FIG. 18, the point cloud encoding device 100 according to the present embodiment will be described. FIG. 18 is a diagram showing an example of the functional blocks of the point cloud encoding device 100 according to the present embodiment.

[0374] As shown in FIG. 18, the point cloud encoding device 100 includes a coordinate conversion unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information encoding unit 1050, a geometric information reconstruction unit 1060, a color conversion unit 1070, an attribute transfer unit 1080, a RAHT unit 1090, a LoD calculation unit 1100, a lifting unit 1110, an attribute information quantization unit 1120, an attribute information encoding unit 1130, and a frame buffer 1140.

[0375] The coordinate conversion unit 1010 is configured to perform conversion processing from the three-dimensional coordinate system of the input point cloud to an arbitrary different coordinate system. For coordinate conversion, for example, by rotating the input point cloud, the x, y, and z coordinates of the input point cloud may be converted into arbitrary s, t, and u coordinates. Also, as one variation of the conversion, the coordinate system of the input point cloud may be used as it is.

[0376] The geometric information quantization unit 1020 is configured to perform quantization of the position information of the input point cloud after coordinate conversion and removal of points with overlapping coordinates. Note that when the quantization step size is 1, the position information of the input point cloud coincides with the quantized position information. That is, when the quantization step size is 1, it is equivalent to the case where quantization is not performed.

[0377] The tree analysis unit 1030 is configured to generate an occupancy code indicating in which node of the encoding target space a point exists based on the tree structure described later, with the position information of the quantized point cloud as input.

[0378] In this process, the tree analysis unit 1030 is configured to generate a tree structure by recursively dividing the encoding target space with rectangular parallelepipeds.

[0379] Here, when points exist within a certain rectangular parallelepiped, a tree structure can be generated by recursively executing the process of dividing such a rectangular parallelepiped into a plurality of rectangular parallelepipeds until the rectangular parallelepiped reaches a predetermined size. Each such rectangular parallelepiped is referred to as a node. Also, each rectangular parallelepiped generated by dividing a node is called a child node, and the occupancy code is a representation of whether or not a point is included in the child node, expressed as 0 or 1.

[0380] As described above, the tree analysis unit 1030 is configured to generate an occupancy code while recursively dividing nodes until they reach a predetermined size.

[0381] In this embodiment, a method called "Octree" that always uses the above-mentioned rectangular parallelepiped as a cube and recursively performs octree division, and a method called "QtBt" that performs quadtree division and binary tree division in addition to octree division can be used.

[0382] Here, whether or not to use "QtBt" is transmitted to the point cloud decoder 200 as control data.

[0383] Alternatively, it may be specified to use Predictive geometry coding using an arbitrary tree structure. In such a case, the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoder 200 as control data.

[0384] For example, the control data of the tree structure may be configured to be decodable by the procedure described in FIGS. 5 to 14.

[0385] The approximate surface analysis unit 1040 is configured to generate approximate surface information using the tree information generated by the tree analysis unit 1030.

[0386] The approximate surface information is, for example, when decrypting the 3D point cloud data of an object, etc., in the case where the point cloud is densely distributed on the object surface, instead of decrypting each individual point cloud, it is the expression obtained by approximating the existence region of the point cloud with a small plane.

[0387] Specifically, the approximate surface analysis unit 1040 may be configured to generate approximate surface information, for example, by a method called "Trisoup". Also, when decrypting sparse point clouds obtained by Lidar or the like, this process can be omitted.

[0388] The geometric information encoding unit 1050 is configured to encode the syntax such as the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040 to generate a bit stream (geometric information bit stream). Here, the bit stream may include, for example, the syntax described in FIG. 4.

[0389] The encoding process is, for example, context-adaptive binary arithmetic encoding. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0390] The geometric information reconstruction unit 1060 is configured to reconstruct the geometric information (the coordinate system assumed by the encoding process, that is, the position information after coordinate transformation in the coordinate transformation unit 1010) of each point of the point cloud data to be encoded based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040.

[0391] The frame buffer 1140 is configured to input the geometric information reconstructed by the geometric information reconstruction unit 1060 and save it as a reference frame.

[0392] The saved reference frame is read from the frame buffer 1140 and used as a reference frame when the tree analysis unit 1030 performs inter-prediction of frames that are temporally different.

[0393] Here, for each frame, which reference frame at which time is used is determined based on, for example, the value of a cost function representing the coding efficiency, and the information of the reference frame to be used may be transmitted to the point cloud decoding device 200 as control data.

[0394] The color conversion unit 1070 is configured to perform color conversion when the input attribute information is color information. The color conversion does not necessarily have to be executed, and whether to execute the color conversion process is encoded as part of the control data and transmitted to the point cloud decoding device 200.

[0395] The attribute transfer unit 1080 is configured to correct the attribute value so that the distortion of the attribute information is minimized based on the position information of the input point cloud, the position information of the point cloud after reconstruction in the geometric information reconstruction unit 1060, and the attribute information after color change in the color conversion unit 1070. As a specific correction method, for example, the method described in Non-Patent Document 1 can be applied.

[0396] The RAHT unit 1090 takes as input the attribute information after transfer by the attribute transfer unit 1080 and the geometric information generated by the geometric information reconstruction unit 1060, and is configured to generate residual information for each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform).

[0397] The information to be decoded is the direct current component (DC coefficient) and the alternating current component (AC coefficient) of the attribute information generated by using RAHT in the encoding process, and in the decoding process, it is converted into attribute information by using the inverse transform of RAHT.

[0398] As a specific process of RAHT, for example, the method described in the above Non-Patent Document 1 can be used.

[0399] The LoD calculation unit 1100 is configured to generate LoD (Level of Detail) using the geometric information generated by the geometric information reconstruction unit 1060 as input.

[0400] LoD is information for defining a reference relationship (points to be referred to and points to be referenced) for realizing predictive coding such as predicting the attribute information of one point from the attribute information of a certain point and encoding or decoding the prediction residual.

[0401] In other words, LoD is information that defines a hierarchical structure in which each point included in the geometric information is classified into a plurality of levels, and for points belonging to a lower level, the attribute is encoded or decoded using the attribute information of points belonging to a higher level.

[0402] As a specific method for determining LoD, for example, the method described in Non-Patent Document 1 above may be used.

[0403] The lifting unit 1110 is configured to generate residual information by lifting processing using the LoD generated by the LoD calculation unit 1100 and the attribute information after attribute transfer in the attribute transfer unit 1080.

[0404] As a specific process of lifting, for example, the method described in Non-Patent Document 1 above may be used.

[0405] The attribute information quantization unit 1120 is configured to quantize the residual information output from the RAHT unit 1090 or the lifting unit 1110. Here, when the quantization step size is 1, it is equivalent to the case where quantization is not performed.

[0406] The attribute information encoding unit 1130 is configured to perform an encoding process using the quantized residual information output from the attribute information quantization unit 1120 as syntax and generate a bit stream related to the attribute information (attribute information bit stream).

[0407] The symbolization process is, for example, a context-adaptive binary arithmetic symbolization process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0408] The point cloud encoding device 100 is configured to perform an encoding process using the position information and attribute information of each point in the point cloud as inputs through the above processes, and output a geometric information bit stream and an attribute information bit stream.

[0409] In addition, the above-described point cloud encoding device 100 and point cloud decoding device 200 may be realized by a program that causes a computer to execute each function (each process).

[0410] In each of the above embodiments, the present invention has been described by taking the application to the point cloud encoding device 100 and the point cloud decoding device 200 as an example. However, the present invention is not limited to such examples only, and can be similarly applied to a point cloud encoding / decoding system having each function of the point cloud encoding device 100 and the point cloud decoding device 200.

Industrial Applicability

[0411] According to the present embodiment, for example, since it is possible to improve the overall service quality in video communication, it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization, and foster innovation."

Explanation of Signs

[0412] 10…Point cloud processing system 100…Point cloud encoding device 1010…Coordinate conversion unit 1020…Geometric information quantization unit 1030…Tree analysis unit 1040…Approximate surface analysis unit 1050…Geometric information encoding unit 1060…Geometric information reconstruction unit 1070…Color conversion unit 1080…Attribute transfer unit 1090…RAHT unit 1100…LoD calculation unit 1110…Lifting unit 1120…Attribute information quantization unit 1130…Attribute information encoding unit 200…Point cloud decoding device 2010…Geometric information decoding unit 2020…Tree synthesis unit 2030…Approximate surface synthesis unit 2040…Geometric information reconstruction unit 2050…Inverse coordinate conversion unit 2060…Attribute information decoding unit 2070…Inverse quantization unit 2080…RAHT unit 2090…LoD calculation unit 2100…Inverse lifting unit 2110…Inverse color conversion unit

Claims

1. A point cloud decoding device, comprising: A point cloud decoding device characterized by comprising an RAHT unit that searches for adjacent nodes in a higher hierarchy in intra prediction, sets a predetermined search range for the search, and determines whether the adjacent node to be searched for exists within the search range or whether there is a possibility that the adjacent node exists within the search range based on the Morton code of the node stored at the start point or end point of the search range.

2. A point cloud decoding method, comprising: A point cloud decoding method comprising the steps of searching for an adjacent node in a higher hierarchy in intra prediction, setting a predetermined search range for the search, and determining whether the adjacent node to be searched for exists within the search range or whether there is a possibility that the adjacent node exists within the search range based on the Morton code of the node stored at the start point or end point of the search range.

3. A program that causes a computer to function as a point group decoding device, The point cloud decoding device is characterized by having a RAHT unit that searches for adjacent nodes in a higher hierarchy in intra prediction, sets a predetermined search range for the search, and determines whether the adjacent node to be searched for exists within the search range or is likely to exist based on the Morton code of the node stored at the start or end point of the search range.

Images