Point cloud decoding device, point cloud decoding method, and program

Abstract

To improve efficiency of decoding attribute information.SOLUTION: In a point cloud decoding device 200 of the present invention, an RAHT unit 2080 is configured to perform scaling on intra-predicted attribute values or AC coefficients obtained by applying RAHT to the attribute values in decoding processing of the attribute values using RAHT.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a point cloud decoding apparatus, a point cloud decoding method, and a program.

Background Art

[0002] As a conventional technique, in decoding attribute information using RAHT, in intra prediction of attribute values, the attribute values of decoded parent nodes, adjacent nodes of the parent node, or adjacent nodes in the same layer are referred to, and weighting is performed according to the adjacent method to predict the attribute value of the processing target node.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technique, since the DC coefficient in the upper layer referred to by the intra prediction value is the average value of its child layer nodes, there is a problem that it is difficult to accurately predict the original attribute value of the processing target node.

[0005] Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide a point cloud decoding apparatus, a point cloud decoding method, and a program capable of improving the coding efficiency of attribute information coding.

Means for Solving the Problems

[0006] A first feature of the present invention is a point cloud decoding device including a RAHT unit, and the gist of the RAHT unit is to perform scaling on an intra-predicted attribute value or an AC coefficient obtained by performing RAHT on the attribute value in the decoding process of the attribute value using RAHT.

[0007] A second feature of the present invention is a point cloud decoding device including a RAHT unit, and the gist of the RAHT unit is to select a reference frame for bidirectional prediction from within the list based on a list of prepared reference frames and the value of the index in the decoded list for each order of the frames processed within the group.

[0008] A third feature of the present invention is a point cloud decoding device including a tree synthesis unit, and the gist of the tree synthesis unit is to select a reference frame for bidirectional prediction from within the list based on a list of prepared reference frames and the value of the index in the decoded list for each order of the frames processed within the group in the prediction process of the position information.

[0009] A fourth feature of the present invention is a point cloud decoding device including a tree synthesis unit, and the gist of the tree synthesis unit is to create one predictor based on the predictor index of each decoded reference frame from among the two selected frames when performing bidirectional prediction in the prediction process of the position information.

[0010] A fifth feature of the present invention is a point cloud decoding method having a step of performing scaling on an intra-predicted attribute value or an AC coefficient obtained by performing RAHT on the attribute value in the decoding process of the attribute value using RAHT.

[0011] The sixth feature of the present invention is a program that causes a computer to function as a point cloud decoding device, the point cloud decoding device including a RAHT unit, and the RAHT unit being configured to perform scaling on an intra-predicted attribute value or an AC coefficient obtained by RAHTing the attribute value in the decoding process of attribute values using RAHT.

Advantages of the Invention

[0012] According to the present invention, it is possible to provide a point cloud decoding device, a point cloud decoding method, and a program that can improve the coding efficiency of encoding of attribute information.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be replaced with existing components, etc. as appropriate, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0015] (First Embodiment) Hereinafter, with reference to FIGS. 1 to 20, a point cloud processing system 10 according to the first embodiment of the present invention will be described. FIG. 1 is a diagram showing the point cloud processing system 10 according to the present embodiment.

[0016] As shown in FIG. 1, the point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0017] The point cloud encoding device 100 is configured to generate encoded data (bit stream) by encoding an input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bit stream.

[0018] Note that the input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud. The attribute information is, for example, color information or reflectivity of each point.

[0019] Here, such a bit stream may be transmitted from the point cloud encoding device 100 to the point cloud decoding device 200 via a transmission path. Also, the bit stream may be stored in a storage medium and then provided from the point cloud encoding device 100 to the point cloud decoding device 200.

[0020] (Point Cloud Decoding Device 200) Hereinafter, with reference to FIG. 2, the point cloud decoding device 200 according to the present embodiment will be described. FIG. 2 is a diagram showing an example of the functional blocks of the point cloud decoding device 200 according to the present embodiment.

[0021] As shown in FIG. 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate conversion unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, a LoD calculation unit 2090, an inverse lifting unit 2100, an inverse color conversion unit 2110, and a frame buffer 2120.

[0022] The geometric information decoding unit 2010 is configured to input a bit stream related to geometric information (geometric information bit stream) among the bit streams output from the point cloud encoding device 100 and decode the syntax.

[0023] The decoding process is, for example, context-adaptive binary arithmetic decoding. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0024] The tree synthesis unit 2020 is configured to generate tree information indicating in which region of the decoding target space points exist, taking as input the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating in which node of the tree described later the point cloud exists.

[0025] Note that the decoding process of the occupancy code may be configured to be performed inside the tree synthesis unit 2020.

[0026] This process can generate tree information by dividing the decoding target space into rectangular parallelepipeds, determining whether points exist in each rectangular parallelepiped with reference to the occupancy code, dividing the rectangular parallelepiped in which points exist into a plurality of rectangular parallelepipeds, and recursively repeating the process of referring to the occupancy code.

[0027] Here, when decoding such an occupancy code, inter prediction described later may be used.

[0028] In this embodiment, a method called "Octree" that recursively performs octree division with the above rectangular parallelepiped always as a cube, and a method called "QtBt" that performs quadtree division and binary tree division in addition to octree division can be used. Whether to use "QtBt" is transmitted from the point cloud encoding device 100 side as control data.

[0029] Alternatively, when specified to use Predictive geometry coding by control data, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree configuration determined in the point cloud encoding device 100.

[0030] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree synthesis unit 2020 and decode the point cloud based on such approximate surface information.

[0031] The approximate surface information is, for example, when the point cloud is densely distributed on the object surface when decoding the three-dimensional point cloud data of the object, etc., instead of decoding each individual point cloud, it is an approximation of the existence region of the point cloud represented by small planes.

[0032] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information and decode the point cloud by, for example, a method called "Trisoup". Specific processing examples of "Trisoup" will be described later. Also, when decoding a sparse point cloud acquired by Lidar or the like, this process can be omitted.

[0033] The geometric information reconstruction unit 2040 is configured to reconstruct the geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point cloud data to be decoded based on the tree information generated by the tree synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0034] The inverse coordinate conversion unit 2050 is configured to input the geometric information reconstructed by the geometric information reconstruction unit 2040, perform conversion from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

[0035] The frame buffer 2120 is configured to input the geometric information reconstructed by the geometric information reconstruction unit 2040 and store it as a reference frame. The stored reference frame is read from the frame buffer 2130 and used as a reference frame when the tree synthesis unit 2020 performs inter-prediction of frames with different times.

[0036] Here, which reference frame at which time is used for each frame may be determined based on, for example, control data transmitted as a bit stream from the point cloud encoding device 100.

[0037] The attribute information decoding unit 2060 is configured to input a bit stream related to attribute information (attribute information bit stream) among the bit streams output from the point cloud encoding device 100 and decode the syntax.

[0038] The decoding process is, for example, context-adaptive binary arithmetic decoding. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0039] Also, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

[0040] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and a quantization parameter that is one of the control data decoded by the attribute information decoding unit 2060, and generate inverse quantized residual information.

[0041] The inverse quantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090 according to the characteristics of the point cloud to be decoded. Which one it is output to is specified by the control data decoded by the attribute information decoding unit 2060.

[0042] The RAHT unit 2080 takes as input the inverse quantized residual information generated by the inverse quantization unit 2070 and the geometric information generated by the geometric information reconstruction unit 2040, and is configured to decode the attribute information of each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform, and inverse Haar transform in the decoding process). The information to be decoded is the direct current component (DC coefficient) and the alternating current component (AC coefficient) of the attribute information generated by using RAHT in the encoding process, and is converted into attribute information by using the inverse transform of RAHT in the decoding process. As a specific process of RAHT, for example, the method described in Non-Patent Document 1 can be used.

[0043] The LoD calculation unit 2090 takes as input the geometric information generated by the geometric information reconstruction unit 2040 and is configured to generate LoD (Level of Detail).

[0044] LoD is information for defining a reference relationship (the points to be referenced and the points being referenced) for realizing predictive coding such as predicting the attribute information of one point from the attribute information of another point and encoding or decoding the prediction residual.

[0045] In other words, LoD is information that defines a hierarchical structure in which each point included in the geometric information is classified into a plurality of levels, and for the points belonging to the lower levels, the attribute information of the points belonging to the upper levels is used to encode or decode the attributes.

[0046] As a specific method for determining LoD, for example, the method described in the above Non-Patent Document 1 may be used.

[0047] The inverse lifting unit 2100 is configured to decode the attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse quantized residual information generated by the inverse quantization unit 2070. As a specific process of inverse lifting, for example, the method described in Non-Patent Document 1 above can be used.

[0048] When the attribute information to be decoded is color information and color conversion has been performed on the point cloud encoding device 100 side, the inverse color conversion unit 2110 is configured to perform inverse color conversion processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100. Whether to execute such inverse color conversion processing is determined by the control data decoded by the attribute information decoding unit 2060.

[0049] The point cloud decoding device 200 is configured to decode and output the attribute information of each point in the point cloud by the above processing.

[0050] (Geometric information decoding unit 2010) Hereinafter, the control data decoded by the geometric information decoding unit 2010 will be described with reference to FIGS. 3 to 4.

[0051] FIG. 3 is an example of the configuration of the encoded data (bit stream) received by the geometric information decoding unit 2010.

[0052] First, the bit stream may include GPS2011. GPS2011 is also called a geometry parameter set and is a set of control data related to the decoding of geometric information. Specific examples will be described later. Each GPS2011 includes at least GPS id information for identifying each individual when there are multiple GPS2011s.

[0053] Second, the bitstream may include GSH2012A / 2012B. GSH2012A / 2012B, also called a geometry slice header or a geometry data unit header, is a set of control data corresponding to a slice described below. Hereinafter, the description will use the term "slice", but it can also be read as a "data unit". Specific examples will be described later. GSH2012A / 2012B includes at least GPS id information for specifying the GPS2011 corresponding to each GSH2012A / 2012B.

[0054] Third, the bitstream may include slice data 2013A / 2013B after GSH2012A / 2012B. The slice data 2013A / 2013B includes data encoding geometric information.

[0055] As described above, the bitstream has a configuration in which one GSH2012A / 2012B and GPS2011 correspond to each slice data 2013A / 2013B.

[0056] As described above, in GSH2012A / 2012B, since the GPS id information is used to specify which GPS2011 to refer to, a common GPS2011 can be used for a plurality of slice data 2013A / 2013B.

[0057] In other words, it is not always necessary to transmit GPS2011 for each slice. For example, as shown in Figure 3, a bitstream configuration can be adopted in which GPS2011 is not encoded immediately before GSH2012B and slice data 2013B.

[0058] Note that the configuration in Figure 3 is only an example. As long as each slice data 2013A / 2013B has a configuration in which GSH2012A / 2012B and GPS2011 correspond, elements other than those described above may be added as components of the bitstream.

[0059] For example, as shown in FIG. 3, the bitstream may include a sequence parameter set (SPS) 2001. Similarly, at the time of transmission, it may be shaped into a configuration different from that of FIG. 3. Furthermore, it may be transmitted as a single bitstream by combining with the bitstream decoded by the attribute information decoder 2060 described later.

[0060] FIG. 4 is an example of the syntax configuration of GPS2011.

[0061] Note that the syntax names described below are merely examples. As long as the functions of the syntaxes described below are the same, the syntax names may be different.

[0062] GPS2011 may include GPS id information (gps_geom_parameter_set_id) for identifying each GPS2011.

[0063] Note that the Descriptor column in FIG. 4 indicates how each syntax is encoded. ue(v) means that it is an unsigned 0th order exponential Golomb code, and u(1) means that it is a 1-bit flag.

[0064] GPS2011 may include a flag (geom_tree_type) for controlling the tree type in the tree synthesis unit 2020.

[0065] For example, when the value of geom_tree_type is "1", it may be defined to use Predictive geometry coding, and when the value of geom_tree_type is "0", it may be defined to use Octree.

[0066] GPS2011 may include a flag (geom_angular_enabled) for controlling whether to perform processing in the Angular mode in the tree synthesis unit 2020.

[0067] For example, when the value of geom_angular_enabled is "1", it may be defined that the process of Predictive geometry coding is performed as the Angular mode, and when the value of geom_angular_enabled is "0", it may be defined that the process of Predictive geometry coding is not performed as the Angular mode.

[0068] GPS2011 may include, in the tree synthesis unit 2020, a flag (ptree_ang_azimuth_scaling_enabled) for controlling whether it is an adaptive azimuth quantization mode in the Angular mode. The adaptive azimuth quantization mode is a mode in which adaptive azimuth quantization according to the radius is performed.

[0069] For example, when the value of ptree_ang_azimuth_scaling_enabled is "1", it may be defined that adaptive azimuth quantization according to the radius is performed, and when the value of ptree_ang_azimuth_scaling_enabled is "0", it may be defined that adaptive azimuth quantization according to the radius is not performed.

[0070] Also, in the calculation (selection) of the predictor in the Angular mode, it may be used as a flag for controlling whether to use the predictor list.

[0071] For example, when the value of ptree_azimuth_scaling_enabled is "1", it may be defined that the predictor list is used in such predictor calculation, and when the value of ptree_ang_azimuth_scaling_enabled is "0", it may be defined that the predictor list is not used in such predictor calculation.

[0072] GPS2011 may include a value (ptree_ang_azimuth_step_minus1) related to the rotation speed of the laser for use in calculating the predicted value of the azimuth angle in the Angular mode in the tree synthesis unit 2020.

[0073] (Tree synthesis unit 2020) Hereinafter, an example of the operation of the tree synthesis unit 2020 will be described with reference to FIGS. 15 to 19.

[0074] FIG. 17 is a flowchart showing an example of the processing in the tree synthesis unit 2020. Hereinafter, an example of the case of synthesizing a tree using "Predictive coding" will be described.

[0075] Predictive coding is also called Predictive geometry coding, Predictive geometry, or Predictive Tree.

[0076] Predictive geometry coding is a means of decoding the predicted position information based on an arbitrary tree structure determined on the point cloud encoding device 100 side and the residual of the position information of the point cloud data, and adding the two to decode the position information of the point cloud data.

[0077] As shown in FIG. 17, in step S501, the tree synthesis unit 2020 determines whether to use inter prediction based on the value of interprediction_enabled_flag.

[0078] If inter prediction is used, the tree synthesis unit 2020 proceeds to step S502, and if inter prediction is not used, the tree synthesis unit 2020 proceeds to step S505.

[0079] In step S502, the tree synthesis unit 2020 acquires a reference frame from the frame buffer 2120.

[0080] The frame buffer 2120 may store one previously decoded frame, and the addition of the decoded frame to the frame buffer 2120 may be performed each time the decoding of one or a specified number of frames is completed. After obtaining the reference frame, the tree synthesis unit 2020 proceeds to step S503.

[0081] In step S503, the tree synthesis unit 2020 determines whether to perform global motion compensation based on the global_motion_enabled_flag.

[0082] If global motion compensation is to be performed, the tree synthesis unit 2020 proceeds to step S504. If global motion compensation is not to be performed, the tree synthesis unit 2020 proceeds to step S505.

[0083] In step S504, the tree synthesis unit 2020 performs global motion compensation on the reference frame obtained in step S502.

[0084] Global motion compensation is a process for correcting the global positional displacement for each frame. For all or a specified range of point groups in the reference frame, rotation and translation based on the global motion vectors decoded by the geometric information decoding unit 2010 are applied. After global motion compensation, the tree synthesis unit 2020 proceeds to step S505.

[0085] In step S505, the tree synthesis unit 2020 decodes the slice data. The specific processing of step S505 will be described later. After decoding the slice data, the tree synthesis unit 2020 proceeds to step S506.

[0086] In step S506, the tree synthesis unit 2020 ends the processing.

[0087] Note that the processing of steps S503 and S504, that is, the determination and execution of global motion compensation, may be performed within the slice data decoding process of step S505.

[0088] FIG. 15 is a flowchart showing an example of the process of slice data decoding in step S505 described above.

[0089] As shown in FIG. 15, in step S1601, the tree synthesis unit 2020 determines whether the decoding of the position information of all the point cloud data included in the slice is completed.

[0090] For this process, for example, information indicating the number of point cloud data included in the slice is transmitted to the GSH, and by comparing this number of point cloud data with the number of data already processed, it can be determined whether the processing of all points is completed.

[0091] If the decoding of the position information of all the point cloud data is completed, this operation proceeds to step S1613 and the process ends. If the decoding of the position information of all the point cloud data is not completed, this operation proceeds to step S1602.

[0092] In step S1602, the tree synthesis unit 2020 sets the parent node of the node to be decoded (the node to be processed) of the point cloud data.

[0093] For example, the tree synthesis unit 2020 decodes the number of child nodes of each node to be decoded, and stores the index of the node to be decoded for the number of such child nodes.

[0094] Then, when the tree synthesis unit 2020 performs the processing of the node to be decoded after a certain node, it refers to the array of the indexes of the node, obtains one index stored at the end of such an array, and may set the node of the obtained index as the parent node of the node to be decoded.

[0095] After the setting of such a parent node is completed, this operation proceeds to step S1603.

[0096] In step S1603, the tree synthesis unit 2020 determines whether to perform processing in Angular mode.

[0097] For example, the tree synthesis unit 2020 can refer to the value of geom_angular_enabled described above to determine whether to perform processing in Angular mode.

[0098] If processing is to be performed in Angular mode, this operation proceeds to step S1604; if not, this operation proceeds to step S1610.

[0099] In step S1604, the tree synthesis unit 2020 decrypts the predictor information and the spherical coordinate residuals to be used in step S1605. Here, the spherical coordinate residuals indicate the residuals of the radius, azimuth angle, and laser ID. When such decryption is completed, this operation proceeds to step S1605.

[0100] In step S1605, the tree synthesis unit 2020 predicts the position information based on the predictor information decrypted in step S1604. Here, the predictor information is the predictor index or the prediction mode. The specific method of such position information prediction will be described later.

[0101] After such position information prediction is completed, this operation proceeds to step S1606.

[0102] In step S1606, the tree synthesis unit 2020 reconstructs the spherical coordinates. In such processing, the tree synthesis unit 2020 reconstructs the spherical coordinates by adding the decrypted spherical coordinate residuals and the predictor.

[0103] After such reconstruction is completed, this operation proceeds to step S1607.

[0104] In step S1607, the tree synthesis unit 2020 performs reconstruction of orthogonal integer coordinates. In such processing, the tree synthesis unit 2020 can perform conversion from the reconstructed spherical coordinates to orthogonal integer coordinates of spherical coordinates. As a specific method, for example, it can be realized by the method described in Non-Patent Document 1.

[0105] After the reconstruction of such orthogonal integer coordinates is completed, this operation proceeds to step S1608.

[0106] In step S1608, the tree synthesis unit 2020 decodes the orthogonal integer coordinate residual.

[0107] After the decoding of the orthogonal integer coordinate residual is completed, this operation proceeds to step S1609.

[0108] In step S1609, the tree synthesis unit 2020 reconstructs the original coordinates. In such processing, the tree synthesis unit 2020 reconstructs the original coordinates by adding the decoded orthogonal integer coordinate residual and the reconstructed orthogonal integer coordinates.

[0109] After the reconstruction of such original coordinates is completed, this operation returns to step S1601.

[0110] In step S1610, the tree synthesis unit 2020 predicts the position information. Specifically, the tree synthesis unit 2020 selects a predictor and uses such predictor as the predicted value of the position information.

[0111] For example, the tree synthesis unit 2020 may select a predictor based on the decoded predictor mode from among a plurality of predictors calculated based on the tree structure.

[0112] After such prediction of the position information is completed, this operation proceeds to step S1611.

[0113] In step S1611, the tree synthesis unit 2020 decodes the orthogonal integer coordinate residual.

[0114] After the decoding of the orthogonal integer coordinate residuals is completed, this operation proceeds to step S1612.

[0115] In step S1612, the tree synthesis unit 2020 reconstructs the original coordinates. In such a process, the tree synthesis unit 2020 reconstructs the original coordinates by adding the orthogonal integer coordinate residuals decoded in step S1611 and the position information predicted in step S1610.

[0116] After the reconstruction of the original coordinates is completed, this operation returns to step S1601.

[0117] FIG. 18 is a flowchart showing an example of the process of predicting position information in step S1605 described above.

[0118] As shown in FIG. 18, in step S701, the tree synthesis unit 2020 decodes the predictor flag.

[0119] Here, the slice data may include a flag indicating the predictor to be used for each node. For example, the slice data may include a flag indicating whether it is an inter predictor or an intra predictor, or a flag similar to the content described in Non-Patent Document 1 and Non-Patent Document 2, such as the index of the inter predictor. Alternatively, the slice data may include other flags described later.

[0120] After decoding the predictor flag, the tree synthesis unit 2020 proceeds to step S702.

[0121] In step S702, the tree synthesis unit 2020 determines whether to use an inter predictor based on the flag decoded in step S701.

[0122] If using an inter predictor, the tree synthesis unit 2020 proceeds to step S704. If not using an inter predictor, the tree synthesis unit 2020 proceeds to step S703.

[0123] In step S703, the tree synthesis unit 2020 performs intra prediction on the coordinates of the node to be processed.

[0124] Here, in such intra prediction, the tree synthesis unit 2020 constructs a predictor based on the coordinates of the parent or ancestor node (e.g., the parent node of the parent node, etc.) of the node to be processed, and predicts the coordinates of the node to be processed.

[0125] In the process of step S703, first, the tree synthesis unit 2020 determines the type of predictor to be used for prediction.

[0126] For example, the tree synthesis unit 2020 may determine whether it is in the adaptive azimuth quantization mode based on the value of ptree_ang_azimuth_scaling_enabled, and determine the type of predictor to be used.

[0127] As the type of predictor, for example, in the case of the adaptive azimuth quantization mode, the tree synthesis unit 2020 may select a predictor to be used based on the decoded prediction mode from among a plurality of predictors calculated using a tree structure.

[0128] Alternatively, in the case of the adaptive azimuth quantization mode, the tree synthesis unit 2020 may hold the position information of the decoded node in a list as a predictor, refer to the one corresponding to the index of the decoded predictor from the list, and select the predictor to be used.

[0129] After the type of predictor to be used is determined, the tree synthesis unit 2020 sets such a predictor as the predicted value of the position information.

[0130] After the intra prediction is completed, the tree synthesis unit 2020 proceeds to step S705.

[0131] In step S704, the tree synthesis unit 2020 performs inter prediction on the coordinates of the node to be processed.

[0132] In such inter prediction, the tree synthesis unit 2020 selects, as a predictor, a node corresponding to the node to be processed from the reference frame, and sets the coordinates of the selected predictor as the predicted value of the coordinates of the node to be processed. A method for selecting a predictor from the reference frame will be described later.

[0133] After the inter prediction is completed, the tree synthesis unit 2020 proceeds to step S705.

[0134] In step S705, the tree synthesis unit 2020 ends the process of step S1605.

[0135] FIG. 19 is a diagram showing an example of a process of selecting a predictor from a reference frame in step S704. However, in the example of FIG. 19, it is assumed that the Angular mode is used. In the Angular mode, the point of the parent node of the node to be processed may be considered to have been decoded immediately before or earlier.

[0136] In FIG. 19, for the parent node of the node to be processed, nodes having the same laser ID and a larger azimuth angle are searched from the reference frame, and the two nodes with the smallest azimuth angles among them are set as predictor 1 and predictor 2, respectively.

[0137] For example, the tree synthesis unit 2020 may perform bidirectional prediction. Hereinafter, an example of the operation of the tree synthesis unit 2020 when performing bidirectional prediction will be described.

[0138] First, the tree synthesis unit 2020 may group a fixed number of frames to be processed and process them after changing the processing order within the group.

[0139] For example, the tree synthesis unit 2020 regards 8 frames as one group, and processes the frames from frame index 0 to 7 within the group in the order of 0, 7, 1, 2, 3, 4, 5, 6.

[0140] Here, the intra-group frame index is a number assigned for each order of frames to be processed within a group.

[0141] Also, there may be two reference frames during inter prediction for each frame to be processed, and the frames to be referenced may be frames in the future in time series.

[0142] The intra-group frame index order pattern and the frames referenced by each intra-group frame index may be decoded as flags included in APS2611 or ASH2612.

[0143] Here, the intra-group frame index order pattern is the pattern of the order of intra-group frame indices.

[0144] When performing bidirectional prediction, the tree synthesis unit 2020, for example, searches for nodes with the same laser ID and a larger azimuth angle as the parent node of the node to be processed from two reference frames, and among them, selects the two with the minimum azimuth angle from each reference frame as predictors, respectively, and generates a total of four predictors. Then, the tree synthesis unit 2020 may use them as predictors based on the decoded predictor index.

[0145] Furthermore, the tree synthesis unit 2020 may prepare a list of reference frames for the frames referenced by each intra-group frame index, and select the frames to be referenced from within such a list based on the value of the decoded index within the list.

[0146] Note that the tree synthesis unit 2020 may prepare two lists of frames in the past and future in time series for the list of reference frames, viewed from the processing frame, and they may be updated at the timing of processing each frame.

[0147] In addition, the tree synthesis unit 2020 may fix and hard-code the frames referred to by each in-group frame index for each in-group frame index order pattern.

[0148] For example, when performing bilateral prediction, the tree synthesis unit 2020 may create one predictor from among the two selected frames based on the predictor index of each decoded reference frame.

[0149] Specifically, the tree synthesis unit 2020 may use the linear prediction values of two frames as predictors. That is, the tree synthesis unit 2020 may use the average value of the predictors of two reference frames as a predictor. Here, the tree synthesis unit 2020 may use the azimuth angle and radius as the objects to be predicted. For the azimuth angle, the tree synthesis unit 2020 may also use a value quantized in units of rotational speed. For example, the tree synthesis unit 2020 may assign weights according to the distance between the reference frame and the frame to be processed.

[0150] In the example of FIG. 19, the tree synthesis unit 2020 searches for nodes with the same laser ID and a larger azimuth angle from the reference frames for the parent node of the node to be processed, and selects the two with the smallest azimuth angles as predictor 1 and predictor 2, respectively.

[0151] FIG. 16 is a flowchart showing an example of the intra prediction in step S703 described above.

[0152] As shown in FIG. 16, in step S1701, the tree synthesis unit 2020 determines whether it is in the adaptive azimuth quantization mode based on the value of ptree_ang_azimuth_scaling_enabled.

[0153] If it is in the adaptive azimuth quantization mode, this operation proceeds to step S1702. On the other hand, if it is not in the adaptive azimuth quantization mode, this operation proceeds to step S1703.

[0154] In step S1702, the tree synthesis unit 2020 decodes the predictor index. After the decoding of the predictor index is completed, this operation proceeds to step S1704.

[0155] In step S1703, the tree synthesis unit 2020 decodes the prediction mode. After the decoding of the prediction mode is completed, this operation proceeds to step S1704.

[0156] In step S1704, the tree synthesis unit 2020 decodes the number of azimuth angle steps. After the decoding of the number of azimuth angle steps is completed, this operation proceeds to step S1705.

[0157] In step S1705, the tree synthesis unit 2020 decodes the spherical coordinate residual. The tree synthesis unit 2020 may perform such decoding using the method described in Non-Patent Document 2. After the decoding is completed, this operation proceeds to step S1706 and the process ends.

[0158] (Attribute information decoding unit 2060) Hereinafter, the control data decoded by the attribute information decoding unit 2060 will be described with reference to FIGS. 5 to 6.

[0159] FIG. 5 is an example of the configuration of the encoded data (bit stream) received by the attribute information decoding unit 2060. FIGS. 6 and 7 are examples of the syntax configurations of APS2611 and ASH2612.

[0160] Note that the syntax names described below are merely examples. As long as the functions of the syntaxes described below are the same, the syntax names may be different.

[0161] APS2611 may include APS id information (aps_geom_parameter_set_id) for identifying each APS2611.

[0162] Note that the Descriptor column in FIG. 4 indicates how each syntax is encoded. ue(v) means that it is an unsigned 0th order exponential Golomb code, and u(1) means that it is a 1-bit flag.

[0163] APS2611 may include a flag (attr_coding_type) for controlling whether to output the inverse quantized residual information to either the RAHT unit 2080 or the LoD calculation unit 2090 in the inverse quantization unit 2070.

[0164] For example, when the value of attr_coding_type is "1", it may be defined to output to the LoD calculation unit 2090, and when the value of attr_coding_type is "0", it may be defined to output to the RAHT unit 2080.

[0165] APS2611 may include a flag (raht_prediction_enabled) for controlling whether to perform prediction of attribute information in the RAHT unit 2080.

[0166] For example, when the value of raht_prediction_enabled is "1", it may be defined to perform prediction of attribute information, and when the value of raht_prediction_enabled is "0", it may be defined not to perform prediction of attribute information.

[0167] APS2611 may include a value (raht_prediction_threshold0) indicating the threshold of the number of adjacent nodes of the grandparent node used to determine whether to perform intra prediction of attribute information in the RAHT unit 2080. Here, the grandparent node refers to the parent node's parent node of the node to be processed.

[0168] APS2611 may include a value (raht_prediction_threshold1) indicating the threshold of the number of adjacent nodes of the parent node used to determine whether to perform intra prediction of attribute information in the RAHT unit 2080.

[0169] APS2611 may include values (raht_prediction_intra_eligibility_threshold0) and (raht_prediction_intra_eligibility_threshold1) that indicate a threshold value obtained by dividing or subtracting the predicted value of the DC coefficient of the processing target node used to determine whether to perform intra-prediction of attribute information and the DC coefficient obtained by RAHT conversion in the RAHT unit 2080.

[0170] APS2611 may include a flag (raht_subnode_prediction_enable_flag) that controls whether to use sub-nodes for prediction of attribute information in the RAHT unit 2080.

[0171] For example, when the value of raht_subnode_prediction_enable_flag is "1", it may be defined that sub-nodes are used for prediction of attribute information, and when the value of raht_subnode_prediction_enable_flag is "0", it may be defined that sub-nodes are not used for prediction of attribute information.

[0172] APS2611 may include weight parameters (raht_prediction_weights) when performing intra-prediction of attribute information in the RAHT unit 2080.

[0173] For example, the value of raht_prediction_weights may be defined according to the way of adjacency between the node to be decoded and the adjacent nodes used for intra-prediction.

[0174] APS2611 may include a flag (raht_inter_prediction_enabled) for controlling whether to perform inter-prediction of attribute information in the RAHT unit 2080.

[0175] For example, when the value of raht_inter_prediction_enabled is "1", it may be defined that prediction of attribute information is performed, and when the value of raht_inter_prediction_enabled is "0", it may be defined that prediction of attribute information is not performed.

[0176] APS2611 may include a value (raht_inter_prediction_depth_minus1) indicating a layer for enabling inter-prediction of attribute information in the RAHT unit 2080.

[0177] For example, when raht_inter_prediction_depth_minus1 is "N - 1", inter-prediction may be enabled up to the upper N layers of the Octree structure.

[0178] APS2611 may include a value (raht_send_inter_filters) indicating whether to transmit a scaling factor in the inter-prediction of attribute information.

[0179] For example, when raht_send_inter_filters is "1", it may be defined that a scaling factor in the inter-prediction of attribute information is transmitted, and when raht_send_inter_filters is "0", it may be defined that a scaling factor in the inter-prediction of attribute information is not transmitted.

[0180] APS2611 may include a value (raht_inter_skip_layers) indicating, for the inter-prediction of attribute information, from which layer up to which upper layer from the layer of the root node of the Octree is excluded from the application of inter-prediction scaling. Here, the root node refers to a node in the slice where the Octree division has not been performed even once.

[0181] For example, when raht_inter_skip_layers is "3", it may be defined that inter-prediction is not applied from the first layer to the third layer.

[0182] APS2611 may include a value (raht_enable_code_layer) indicating whether to transmit the applicability mode of inter-prediction for each layer. Alternatively, APS2611 may be considered to include raht_enable_code_layer when raht_prediction_enabled is "1" and raht_inter_prediction_enabled is "1".

[0183] For example, when raht_enable_code_layer is "1", it may be defined that the applicability mode of inter-prediction for each layer is transmitted, and when raht_enable_code_layer is "0", it may be defined that the applicability mode of inter-prediction for each layer is not transmitted.

[0184] APS2611 may include a flag (biPredictionPrediod) indicating the prediction method of the attribute information in the RAHT unit 2080.

[0185] For example, when biPredictionPrediod is "0", the prediction method of the attribute information may be defined as "no prediction" or "intra-prediction", when biPredictionPrediod is "1", the prediction method of the attribute information may be defined as "no prediction" or "intra-prediction" or "inter-prediction", and when biPredictionPrediod is "2", the prediction method of the attribute information may be defined as "no prediction" or "intra-prediction" or "inter-prediction" or "bidirectional prediction".

[0186] The bidirectional prediction will be described later. Note that "no prediction" of the prediction method of the attribute information means that in the RAHT unit 2080, the decoded AC coefficients are directly used for inverse RAHT without performing prediction of the AC coefficients.

[0187] APS2611 may include a value (raht_send_inter_filters_intra) indicating whether to transmit the scaling factor in the intra prediction of attribute information.

[0188] For example, when raht_send_inter_filters_intra is "1", it may be defined that the scaling factor in the intra prediction of attribute information is transmitted, and when raht_send_inter_filters_intra is "0", it may be defined that the scaling factor in the intra prediction of attribute information is not transmitted.

[0189] ASH2612 may include a value (layer_code_depth) indicating the number of applicable modes (raht_attr_layer_code_mode) of inter prediction for each layer described below when raht_inter_prediction_enabled is "1" and raht_enable_code_layer is "1".

[0190] Alternatively, ASH2612 may include layer_code_depth when either raht_enable_code_layer or raht_send_inter_filters is "1".

[0191] Or, for example, ASH2612 may include layer_code_depth only when raht_send_inter_filters is "1".

[0192] Or, layer_code_depth may be defined as a value obtained by subtracting 1 from the number of layers of the frame, and may be used after adding 1 after decoding.

[0193] Or, when layer_code_depth is "0", layer_code_depth may be used as "0", and when it is other than "0", it may be used after subtracting 1 after decoding.

[0194] Alternatively, layer_code_depth may be set to be equal to the smaller value between the value obtained by subtracting 1 from the number of layers of the slice and raht_inter_prediction_depth_minus1. When raht_enable_code_layer is "1", ASH2612 may include the applicability mode of inter prediction (raht_attr_layer_code_mode) for each layer by the number of layer_code_depth.

[0195] For example, in each layer, when inter prediction is applied, it may be defined as "1", and when inter prediction is not applied, it may be defined as "0".

[0196] Alternatively, raht_attr_layer_code_mode may be configured with 3 bits, and the flags indicated by each bit may be defined as follows.

[0197] The first bit may be defined as a value indicating "no prediction" or "with prediction", where when the first bit is "0", it is defined as "no prediction", and when the first bit is "1", it is defined as "with prediction".

[0198] The second bit may be defined as a value indicating "prediction method", where when the second bit is "0", it is defined as "intra prediction", and when the second bit is "1", it is defined as "inter prediction".

[0199] The third bit may be defined as a value indicating "inter prediction method", where when the third bit is "0", it is defined as "inter prediction", and when the third bit is "1", it is defined as "bidirectional prediction".

[0200] Also, according to the value of biPredictionPrediod, the number of bits of raht_attr_layer_code_mode to be decoded may be determined.

[0201] For example, when biPredictionPrediod is "0", it may be assumed that only the first bit is decoded as raht_attr_layer_code_mode.

[0202] Note that, for example, when biPredictionPrediod is "0" and raht_attr_layer_code_mode is "1", it may be defined as intra prediction as the prediction method.

[0203] When biPredictionPrediod is "1", it may be assumed that only the first and second bits are decoded as raht_attr_layer_code_mode.

[0204] When biPredictionPrediod is "2", it may be assumed that the first, second, and third bits are decoded as raht_attr_layer_code_mode.

[0205] When raht_send_inter_filters is "1", ASH2612 may include only the number of scaling factors (num_filter_taps) in the scaling factor in inter prediction and the residual from the scaling factor (raht_filter_taps).

[0206] As shown in FIG. 7, raht_filter_taps may be decoded when raht_attr_layer_code_mode[i + raht_inter_skip_layers - 1] is "1" in the decoding of raht_filter_taps[i].

[0207] Also, the initial value of raht_filter_taps may be defined as "0".

[0208] Also, when raht_attr_layer_code_mode[i + raht_inter_skip_layers - 1] is "0", raht_filter_taps[i] may have an initial value of "0" set.

[0209] Also, in the decoding of raht_filter_taps[i], when raht_inter_skip_layers is "0", an initial value of "0" may be set when i is "0".

[0210] Also, FIG. 14 is an example of the syntax structure when decoding raht_filter_taps based on raht_inter_skip_layers.

[0211] Hereinafter, only the differences from the syntax structure described in FIG. 7 are shown. raht_filter_taps may be decoded even when i is "0" when raht_inter_skip_layers is "0" in the decoding of raht_filter_taps[i].

[0212] num_filter_taps may be derived based on the syntax that specifies the layer to which the decoded inter prediction is applied.

[0213] Hereinafter, an example of a method for deriving num_filter_taps will be described.

[0214] num_filter_taps may be included in ASH2612 when raht_enable_code_layer is "0", or may be derived by the following method when raht_enable_code_layer is "1".

[0215] For example, num_filter_taps may be derived based on a value (raht_inter_skip_layers) indicating up to which upper layer the scaling application of inter prediction is not to be performed, a value (raht_inter_prediction_depth_minus1) indicating the number of effective layers of inter prediction, and a value (layer_code_depth) indicating the number of raht_attr_layer_code_mode.

[0216] Here, the number of effective layers of inter prediction refers to a numerical value indicating the threshold of the layer to which inter prediction is applied. For example, the number of effective layers of inter prediction may be a value obtained by adding 1 to raht_inter_prediction_depth_minus1. When raht_inter_prediction_depth_minus1 is "N - 1", the number of effective layers of inter prediction may be defined as "N".

[0217] Specifically, for example, when the number of layers of the frame is larger than the number of effective layers of inter prediction, num_filter_taps may be obtained by subtracting a value indicating up to which upper layer the scaling application of inter prediction is not to be performed from the number of effective layers of inter prediction. When the value indicating the number of raht_attr_layer_code_mode is smaller than the number of effective layers of inter prediction, num_filter_taps may be obtained by subtracting a value indicating up to which upper layer the scaling application of inter prediction is not to be performed from the value indicating the number of raht_attr_layer_code_mode.

[0218] That is, num_filter_taps may be derived as follows.

[0219] [Number] Alternatively, regardless of the values of raht_inter_skip_layers and raht_inter_prediction_depth_minus1, num_filter_taps may be derived as follows.

[0220] num_filter_taps = layer_code_depth - raht_inter_skip_layers - 1 Alternatively, num_filter_taps may be num_filter_taps = layer_code_depth - raht_inter_skip_layers derived as such, and at this time, in the decoding of raht_filter_taps, when raht_attr_layer_code_mode[i + raht_inter_skip_layers] is "1", raht_filter_taps[i] may be decoded.

[0221] In addition, the attribute information decoding unit 2060 may derive the number of scaling factors using the applicability mode of inter prediction for each layer.

[0222] Specifically, the attribute information decoding unit 2060 may, for example, count the layers to which inter prediction is applied based on the applicability mode of inter prediction for each layer.

[0223] However, the attribute information decoding unit 2060 may exclude from the count the layers for which inter prediction scaling is not applied based on the value indicating up to which upper layer inter prediction scaling is not applied.

[0224] Alternatively, the attribute information decoding unit 2060 may decode the scaling factor only when the layer is a layer to which inter prediction is applied based on the applicability mode of inter prediction for each layer.

[0225] However, the attribute information decoding unit 2060 may not decode the layers for which inter prediction scaling is not applied based on the value indicating up to which upper layer inter prediction scaling is not applied. ASH2612 may include the number of scaling factors (num_filter_taps_intra) in intra prediction when raht_send_inter_filters_intra is "1".

[0226] ASH2612 may include the residual of the scaling factor (raht_filter_taps_intra) by the number of num_filter_taps_intra.

[0227] num_filter_taps_intra may be derived based on the syntax that specifies the layer to which the decoded intra prediction is applied.

[0228] Note that above, an example of decoding the above information in APS2611 has been described. However, such information may be included in ASH2612 or may be included in SPS2601. That is, such information may be included in any header. (RAHT Unit 2080) An example of the processing of the RAHT unit 2080 will be described with reference to FIGS. 8 to 13.

[0229] FIG. 8 is a flowchart showing an example of the processing of the RAHT unit 2080.

[0230] As shown in FIG. 8, in step S28001, the RAHT unit 2080 recursively octree-divides the nodes until they reach a predetermined size using a method called Octree. After such division is completed, this operation proceeds to step S28002.

[0231] In step S28002, the RAHT unit 2080 totals the number of points belonging to the lower layer of the node in each node divided by the Octree.

[0232] Specifically, the RAHT unit 2080 scans the nodes of a certain layer in sequence and records the number of points belonging to each node. Next, the RAHT unit 2080 sums up the number of points recorded in the child nodes of each node in the nodes of the upper layer and calculates the number of points belonging to each node.

[0233] The RAHT unit 2080 repeats the above scanning in sequence from the bottom layer to the top layer. The total number of acquired points is used as the weight for the inverse transformation of RAHT in step S28005 described later. After such calculation is completed, this operation proceeds to step S28003.

[0234] In step S28003, the RAHT unit 2080 decrypts the DC coefficient of the node belonging to the top layer of the Octree. Alternatively, the RAHT unit 2080 may predict the DC coefficient using intra prediction and calculate the DC coefficient by decrypting and adding the prediction residual of the DC coefficient.

[0235] After the decryption of such DC coefficient is completed, the RAHT unit 2080 calculates the attribute value Aroot of the root node using the total number of points wroot belonging to the root node acquired in step S28002 and the decrypted DC coefficient DCroot according to the following formula.

[0236]

Equation

[0237] In step S28004, the RAHT unit 2080 determines whether the decryption of the attribute information of all the nodes included in the layer is completed.

[0238] If not completed, this operation proceeds to step S28005, and if completed, this operation proceeds to step S28007.

[0239] In step S28005, the RAHT unit 2080 decodes the AC coefficients. Specifically, it will be described later. After such decoding is completed, this operation proceeds to step S28006.

[0240] In step S28006, the RAHT unit 2080 calculates the attribute values using the inverse transform of RAHT based on the total number of points belonging to the lower hierarchy of each aggregated node, the decoded AC coefficients, and the DC coefficients calculated from the nodes in the upper hierarchy by the method described later.

[0241] Here, the inverse transform of RAHT is performed in units of 8 nodes of 2×2×2 that are octree-divided.

[0242] Specifically, the attribute values A1, A2, … A k are obtained by the following formula (1) using the DC coefficient DC of the node holding k sub-nodes, the AC coefficients AC1, AC2, … AC k-1 and the total number of points w = w1, w2, … w belonging to the lower hierarchy of each sub-node. k are obtained using the following formula (1).

[0243]

Equation

[0244] It is assumed that such conversion processing is repeatedly performed in the order from the nodes in the upper hierarchy to the nodes in the lower hierarchy.

[0245]

Equation

[0246] In step S28007, the RAHT unit 2080 determines whether the decoding of all nodes in all levels has been completed.

[0247] If not completed, this operation moves the processing target level to the next lower level and proceeds to step S28004. If completed, this operation proceeds to step S28008 and ends the processing.

[0248] FIG. 9 is a flowchart showing an example of the processing in step S28004.

[0249] As shown in FIG. 9, in step S28101, the RAHT unit 2080 determines whether to predict the AC coefficients. When making such a determination, the RAHT unit 2080 may refer to the value of raht_prediction_enabled and use such a value.

[0250] The RAHT unit 2080 may decode a flag indicating whether to perform prediction of AC coefficients in the current processing target node and use the value of such a flag.

[0251] Such a flag may be decoded for each node or for each level. Such a flag may be decoded only when the value of raht_prediction_enabled is the value "1" indicating that prediction is valid. Such a flag may be included in the slice data.

[0252] As a result of the determination, if the AC coefficients are not predicted, this operation proceeds to step S28102, and if the AC coefficients are predicted, this operation proceeds to steps S28103 and S28104.

[0253] In step S28102, the RAHT unit 2080 decodes the AC coefficients. After such decoding is completed, this operation proceeds to step S28113 and ends the processing.

[0254] In step S28107, the RAHT unit 2080 determines whether inter prediction is valid.

[0255] For such determination, the RAHT unit 2080 may refer to the value of raht_inter_prediction_enabled and use such value.

[0256] If the result of the determination is that inter prediction is valid, this operation proceeds to step S28109, and if inter prediction is invalid, this operation proceeds to step S28112.

[0257] In step S28109, the RAHT unit 2080 determines whether the depth of the layer containing the node to be processed is less than or equal to a threshold value. The RAHT unit 2080 may refer to the value of raht_inter_prediction_depth_minus1 as the threshold value and use such value.

[0258] If the result of the determination is that the depth is less than or equal to the threshold value, this operation proceeds to step S28110, and if the depth is greater than the threshold value, this operation proceeds to step S28104.

[0259] In step S28110, the RAHT unit 2080 determines whether to inter predict the AC coefficient of the node to be processed.

[0260] For the determination, the RAHT unit 2080 may check whether inter prediction is executable, and if it is executable, it may not perform inter prediction. Specifically, it will be described later.

[0261] For the determination, the RAHT unit 2080 may decode a flag indicating whether to inter predict the AC coefficient of the node to be processed and use the value of such flag. Such flag may be decoded for each node or for each layer. Such flag may be decoded only when it is determined that inter prediction is executable and may be used for the determination. Such flag may be included in the slice data.

[0262] Such a flag may refer to raht_attr_layer_code_mode and use its value. Such a value may be referred to when the depth of the layer containing the node to be processed is less than layer_code_depth and when the depth of the layer containing the node to be processed is greater than the depth of the root node.

[0263] That is, such a value may be referred to when depth - 1 < layer_code_depth and depth - 1 ≥ 0.

[0264] Here, depth is defined as "0" at the layer of the root node and is a value that is counted up as the layer gets deeper.

[0265] If not referred to, it may be determined that inter prediction is not executable.

[0266] If it is determined that inter prediction is executable, this operation proceeds to step S28111, and if it is determined that inter prediction is not executable, this operation proceeds to step S28104.

[0267] In step S28111, the RAHT unit 2080 performs inter prediction on the AC coefficients of the node to be processed. Specifically, it will be described later.

[0268] In step S28104, the RAHT unit 2080 determines whether to perform intra prediction on the AC coefficients of the node to be processed.

[0269] For example, the RAHT unit 2080 determines whether the number of adjacent nodes of the parent node and grandparent node of the node to be processed is greater than or equal to a threshold. If it is greater than or equal to the threshold, it may be determined to perform intra prediction, and if it is less than the threshold (that is, if it is determined that the accuracy of intra prediction of the AC coefficients is not high), it may be determined not to perform intra prediction.

[0270] The RAHT unit 2080 may refer to the value of the above-mentioned raht_prediction_threshold0 and use such value as the threshold for the adjacent nodes of the grandparent node. Also, it may refer to the value of the above-mentioned raht_prediction_threshold1 and use such value as the threshold for the adjacent nodes of the parent node.

[0271] Alternatively, for the processing target node determined to perform intra prediction in the determination using the above-mentioned raht_prediction_threshold0 and raht_prediction_threshold1, the RAHT unit 2080 may perform an additional determination.

[0272] That is, in step S28104, the RAHT unit 2080 determines the effect of the intra prediction of the AC coefficient of the attribute value using RAHT. In other words, in step S28104, the RAHT unit 2080 determines whether the accuracy of the intra prediction of the AC coefficient of the attribute value using RAHT is high.

[0273] For example, the RAHT unit 2080 may determine whether to perform intra prediction using the DC coefficient (that is, whether the accuracy of the intra prediction of the AC coefficient is high).

[0274] Specifically, when the value obtained by dividing the predicted value of the DC coefficient of the processing target node by the DC coefficient obtained in step S28006 is within the threshold range (that is, when it is determined that the accuracy of the intra prediction of the AC coefficient is high), the RAHT unit 2080 may determine to perform intra prediction. When it is outside the threshold range (that is, when it is determined that the accuracy of the intra prediction of the AC coefficient is not high), the RAHT unit 2080 may determine not to perform intra prediction.

[0275] Alternatively, when the value obtained by subtracting the DC coefficient obtained in step S28006 from the predicted value of the DC coefficient of the node to be processed is within the threshold range (that is, when it is determined that the accuracy of the intra prediction of the AC coefficient is high), the RAHT unit 2080 may determine to perform intra prediction. When it is outside the threshold range (that is, when it is determined that the accuracy of the intra prediction of the AC coefficient is not high), the RAHT unit 2080 may determine not to perform intra prediction.

[0276] The RAHT unit 2080 may use such values as thresholds with reference to the values of raht_prediction_intra_eligibility_threshold0 and raht_prediction_intra_eligibility_threshold1 described above.

[0277] Specifically, when the value obtained by dividing the predicted value of the DC coefficient of the node to be processed by the DC coefficient obtained in step S28006, or the value obtained by subtracting the DC coefficient obtained in step S28006 from the predicted value of the DC coefficient of the node to be processed, is equal to or greater than the value of raht_prediction_intra_eligibility_threshold0 and equal to or less than the value of raht_prediction_intra_eligibility_threshold1 (that is, when it is determined that the accuracy of the intra prediction of the AC coefficient is high), the RAHT unit 2080 may determine to perform intra prediction.

[0278] Here, the predicted value of the DC coefficient is a value obtained simultaneously when converting the predicted value of the attribute value into an AC coefficient in step S28207 described later, and can be obtained by performing the same process in step S28104.

[0279] When it is determined not to perform intra prediction, this operation proceeds to step S28102. When it is determined to perform intra prediction, this operation proceeds to step S28112.

[0280] In step S28112, the RAHT unit 2080 performs intra prediction on the AC coefficients of the node to be processed. Specifically, it will be described later.

[0281] In step S28103, the RAHT 2080 decodes the AC coefficient residual. After such decoding is completed, this operation proceeds to step S28105.

[0282] In step S28105, the RAHT unit 2080 adds the decoded AC coefficient residual and the predicted AC coefficient to perform reconstruction of the AC coefficient. After such reconstruction is completed, this operation proceeds to step S28106 and ends the process.

[0283] Note that the conditional branch in step S28109 may be omitted.

[0284] In the inter prediction process of step S28111, processing equivalent to the intra prediction in step S28112 may also be performed together, and the results of inter prediction and intra prediction may be combined for prediction. Specifically, it will be described later.

[0285] FIG. 10 is a flowchart showing an example of the intra prediction process in step S28112.

[0286] As shown in FIG. 10, in step S28201, the RAHT unit 2080 determines whether to perform intra prediction using adjacent nodes in the subnode hierarchy.

[0287] The RAHT unit 2080 may refer to the value of the raht_subnode_prediction_enable_flag for the determination and use such a value.

[0288] If the RAHT unit 2080 does not use adjacent nodes in the subnode hierarchy, it performs intra prediction using only adjacent nodes in the upper hierarchy.

[0289] Here, the adjacent nodes in the upper layer refer to, among the 19 nodes including the 6 nodes adjacent in terms of surface among the nodes adjacent to the parent node of the node to be decoded, the 12 nodes adjacent in terms of edge, and the parent node itself, the 3 nodes adjacent in terms of surface to the node to be decoded, the 3 nodes adjacent in terms of edge to the node to be decoded, and the 7 nodes including the parent node itself.

[0290] Figure 11 is a diagram showing the relationship between the node to be decoded and the adjacent nodes in the upper layer.

[0291] When using the adjacent nodes in the sub-node layer, the RAHT unit 2080 performs intra prediction using the adjacent nodes in the upper layer and the adjacent nodes in the sub-node layer.

[0292] Here, the adjacent nodes in the sub-node layer are the sub-nodes of the adjacent nodes in the upper layer that are adjacent to the node to be decoded in terms of surface or edge and have been decoded.

[0293] Figure 12 is a diagram showing the relationship between the node to be decoded and the adjacent nodes in the sub-node layer.

[0294] As a result of the determination, when performing intra prediction without using the adjacent nodes in the sub-node layer, this operation proceeds to step S28202, and when performing intra prediction using the adjacent nodes in the sub-node layer, this operation proceeds to step S28204.

[0295] In step S28202, the RAHT unit 2080 acquires the attribute values of the adjacent nodes in the upper layer. After acquiring the attribute values of the adjacent nodes in the upper layer, this operation proceeds to step S28203.

[0296] In step S28203, the RAHT unit 2080 predicts the attribute value of the node to be decoded.

[0297] The RAHT unit 2080 uses the attribute values attr of the k acquired adjacent nodes in the upper layer i and the weight w according to the type of the adjacent node i iUsing the following formula, the prediction of the attribute value attr may be performed.

[0298]

Number

[0299] After the prediction of such an attribute value is completed, this operation proceeds to step S28207.

[0300] In step S28204, the RAHT unit 2080 acquires the attribute values of the adjacent nodes in the upper layer.

[0301] Here, the target for acquiring the attribute value is a node among the adjacent nodes in the upper layer where the sub-nodes of each adjacent node are not yet decoded, or a node among the adjacent nodes in the upper layer where, even if the sub-nodes are decoded, there are no sub-nodes adjacent to the node to be decoded in terms of face or edge.

[0302] After the acquisition of such an attribute value is completed, this operation proceeds to step S28205.

[0303] In step S28205, the RAHT unit 2080 acquires the attribute values of the adjacent nodes in the sub-node layer. After acquiring the attribute values of the adjacent nodes in the sub-node layer, this operation proceeds to step S28206.

[0304] In step S28206, the RAHT unit 2080 predicts the attribute value of the node to be decoded.

[0305] The RAHT unit 2080 uses the acquired attribute values attr of the k upper-layer adjacent nodes and sub-node layer adjacent nodesi and the weight w corresponding to the type i of the adjacent node i Using these, the prediction of the attribute value attr may be performed by the following formula.

[0306] [Number] Here, the RAHT unit 2080 uses the weight w i As, depending on whether the adjacent node i is any of the upper - layer face - adjacent node, upper - layer edge - adjacent node, parent node, sub - node - layer face - adjacent node, and sub - node - layer edge - adjacent node, a hard - coded value may be used, or the value of raht_prediction_weights may be referred to, and the weight w i may be calculated therefrom.

[0307] After the completion of the prediction of such an attribute value, this operation proceeds to step S28207.

[0308] In step S28207, the RAHT unit 2080 converts the predicted attribute value into an AC coefficient. The AC coefficient is generated by performing RAHT on the predicted attribute value. For example, the RAHT unit 2080 may use the method described in Non - Patent Document 1 as such a conversion method.

[0309] For the predicted value AC of the AC coefficient converted by the RAHT unit 2080 intra with respect to the scaling factor α intra it may be multiplied by α intra times.

[0310] AC pred = α intra × AC intra Here, the coefficient α intra may be any real number. The coefficient α intra may be decoded for each node or for each layer. The coefficient α intra may be decoded as a syntax included in APS2611 or ASH2612, or may be included in the slice data. The coefficient αintra It may be hard-coded.

[0311] For example, for the coefficient α intra is defined as follows using the depth of the hierarchy depth, and α intra may be decoded instead of α intra '.

[0312] α intra = 1 + α intra ' × 2 -depth For example, an integer β that takes a range from integer a to integer b is defined, and the RAHT unit 2080 may decode the integer β.

[0313] The RAHT unit 2080 may calculate, for the coefficient α intra as follows, as a value obtained by adding an integer c to the decoded integer β and then dividing by the integer c.

[0314] α intra = (β + c) / c Here, the RAHT unit 2080 may decode the integer β using the exponential Golomb code.

[0315] Alternatively, for example, when the value of the decoded raht_filter_taps_intra is "X", the RAHT unit 2080 subtracts X from 128, and uses the value obtained by right-shifting the subtracted result by 7 bits as the scaling factor α intra for inter prediction.

[0316] For example, when the value of raht_filter_taps_intra is "0", the value of the scaling factor α intra in the inter prediction of the attribute information may be defined as the value "1" obtained by subtracting 0 from 128 and then right-shifting the subtracted result by 7 bits.

[0317] When it is determined that intra prediction is applicable to the node to be processed, for example, by referring to raht_attr_layer_code_mode, the RAHT unit 2080 may scale the intra prediction value using the value of the decoded raht_filter_taps_intra.

[0318] After the conversion of the AC coefficients is completed, this operation proceeds to step S28208 and ends the process.

[0319] FIG. 13 is a diagram showing an example of the inter prediction process of step S28111.

[0320] The RAHT unit 2080 predicts the AC coefficients of the node to be processed using the information of the reference node, which is the corresponding node in the reference frame within the reference frame. Here, the information of the reference node may be its attribute value or AC coefficient. Also, the reference frame refers to another decoded frame, and its information may be included in the previous frame buffer 2120.

[0321] The RAHT unit 2080 may apply the same Octree structure as the frame to be processed to the reference frame. In such a case, there may be a possibility that a node is set at a position where there is no point. Such a node is called an empty node. If the reference node is an empty node, the RAHT unit 2080 may make the inter prediction unexecutable in step S28110.

[0322] The RAHT unit 2080 may apply an Octree independently to the reference frame and set an Octree structure different from that of the frame to be processed. In such a case, there may not necessarily be a node at the same position as the frame to be processed. If the reference node corresponding to the node to be processed is not found, the RAHT unit 2080 may make the inter prediction unexecutable in step S28143.

[0323] When the reference node is an empty node or the reference node cannot be found, the RAHT unit 2080 may estimate and interpolate the information of the reference node using the information of the nodes at the neighboring positions in the reference frame.

[0324] For example, the RAHT unit 2080 may estimate and interpolate the average value of the attribute values or AC coefficients of the adjacent nodes, the nearest neighbor node, or the k-nearest neighbor nodes with respect to the reference node position as the attribute value or AC coefficient of the reference node, respectively.

[0325] Note that the RAHT unit 2080 may apply the above interpolation only after a specific layer.

[0326] When it is determined that it is more efficient in terms of coding not to decode the AC coefficient of the attribute value, the RAHT unit 2080 may skip decoding the AC coefficient of the attribute value of the hierarchical nodes under the processing target node.

[0327] Specifically, when there are two or fewer nodes to be decoded in the parent node including the processing target node, or when the value of the decoded AC coefficient is equal to or less than the threshold value, or when there are two or fewer nodes to be decoded and the value of the decoded AC coefficient is equal to or less than the threshold value, the RAHT unit 2080 determines that it is more efficient in terms of coding not to decode the AC coefficient of the attribute value, and may skip decoding the AC coefficient of the hierarchical nodes under such a processing target node.

[0328] Here, such a threshold value may be a hard-coded value, or the value of raht_prediction_skip_threshold may be referred to and such a value may be used.

[0329] Also, skipping the decoding of the AC coefficient of the hierarchy under the above processing target node may be applied only after a specific layer.

[0330] The RAHT unit 2080 may predict, for example, the AC coefficient of the processing target node from the attribute value of the reference node.

[0331] Specifically, the RAHT unit 2080 uses the value Attr of the decoded attribute value of the reference node inter to obtain the predicted value Attr of the attribute value of the node to be processed pred and applies RAHT to the predicted value Attr of the attribute value of the node to be processed pred to obtain the predicted value AC of the AC coefficient of the node to be processed. pred This may be acceptable.

[0332] Attr pred = Attr inter AC pred = RAHT(Attr pred ) For the AC coefficient of the node to be processed, the RAHT unit 2080 may directly predict it from the AC coefficient of the reference node, for example.

[0333] Specifically, the RAHT unit 2080 calculates the value AC of the AC coefficient of the reference node using RAHT in the reference frame inter and uses that value as the predicted value AC of the AC coefficient of the node to be processed. pred This may be acceptable.

[0334] AC pred = AC inter For the AC coefficient of the reference node, the RAHT unit 2080 may record the AC coefficients of each node in the reference frame in the frame buffer 2120 and obtain them by referring to the values in the frame buffer 2120. In such a case, when the AC coefficient of the reference node does not exist in the frame buffer 2120, the RAHT unit 2080 may set the inter prediction to be infeasible in step S28110.

[0335] Note that the RAHT unit 2080 may multiply Attr inter and AC inter by α times respectively with the scaling factor α.

[0336] Attr pred = α Attr inter Alternatively AC pred = αAC inter The coefficient α may take any real value. The coefficient α may be decoded for each node or for each layer. The coefficient α may be decoded as a syntax included in APS2611 or ASH2612, or may be included in the slice data.

[0337] For example, the coefficient α may be defined using the depth of the layer as follows, and α' may be decoded instead of the coefficient α.

[0338] α = 1 + α'·2 -depth For example, an integer β may be defined as an integer ranging from integer a to integer b, and β may be decoded. The coefficient α may be calculated as a value obtained by adding the integer c to the decoded β and then dividing by the integer c as follows.

[0339] α = (β + c) / c The integer β may be decoded using the exponential Golomb code.

[0340] Alternatively, for example, when the value of the decoded raht_filter_taps is "X", the RAHT unit 2080 may subtract X from 128, right-shift the subtracted result by 7 bits, and use the resulting value as the scaling factor α for the inter prediction.

[0341] For example, when the value of raht_filter_taps is "0", the value of the scaling factor α in the inter prediction of the attribute information may be defined as the value obtained by subtracting 0 from 128 and then right-shifting the subtracted result by 7 bits, which is "1".

[0342] The RAHT unit 2080 may determine whether to apply the inter prediction based on the syntax specifying the layer to which the inter prediction is applied. When it is determined to apply the inter prediction in such a layer, the inter prediction value may be scaled using the value of the decoded raht_filter_taps.

[0343] On the other hand, if it is determined that the scaling of inter prediction is not applied at such a layer, the RAHT unit 2080 may not scale the inter prediction.

[0344] Specifically, if the depth of the layer including the node to be processed is less than or equal to the effective number of layers of inter prediction, and the depth of the layer including the node to be processed is greater than or equal to a value indicating up to which upper layer the scaling of inter prediction is not applied, the RAHT unit 2080 may determine to scale the inter prediction value.

[0345] Here, as the effective number of layers of inter prediction, the RAHT unit 2080 may refer to the value of raht_inter_prediction_depth_minus1 and use such a value.

[0346] Also, as the value indicating up to which upper layer the scaling of inter prediction is not applied, the RAHT unit 2080 may refer to the value of the raht_inter_skip_layers and use such a value.

[0347] Alternatively, for example, if the depth of the layer including the node to be processed is less than or equal to the effective number of layers of inter prediction, and the depth of the layer including the node to be processed is greater than or equal to a value indicating up to which upper layer the scaling of inter prediction is not applied, and it is determined that inter prediction is applied in the layer including the node to be processed, the RAHT unit 2080 may determine to scale the inter prediction value.

[0348] Here, in the layer including the node to be processed, the RAHT unit 2080 may refer to the value of the above-mentioned raht_attr_layer_code_mode and determine whether inter prediction is applied based on such a value.

[0349] Alternatively, when the depth of the hierarchy containing the node to be processed is equal to or greater than a value indicating up to which upper layer the scaling application of inter prediction is excluded, the RAHT unit 2080 may determine to scale the inter prediction value.

[0350] Here, as the value indicating up to which upper layer the scaling application of inter prediction is excluded, the RAHT unit 2080 may refer to the value of the raht_inter_skip_layers and use such value.

[0351] So far, the case where there is one scaling factor for each layer has been described. However, for example, even when the scaling factor is transmitted for each frequency index idx of the AC coefficient, the number of scaling factors to be decoded can be derived by multiplying the number of scaling factors calculated above by the number of scaling factors for each layer. The number of scaling factors may be, for example, 7.

[0352] For example, it may be calculated as follows using the AC coefficient AC of the parent node of the node to be decoded parent and the inter prediction value AC parent_inter when the parent node is decoded.

[0353] α = AC parent / AC parent_inter For example, the RAHT unit 2080 may use the AC coefficients AC neighbor1 AC neighbor2 ... AC neighborN of N adjacent nodes of the node to be decoded and the inter prediction values AC neighbor_inter1 AC neighbor_inter2 ... AC neighbor_interN when each adjacent node is decoded, and calculate α so that the cost is minimized.

[0354] The cost may be, for example, the sum of the squared errors between the AC coefficient of each adjacent node and the AC coefficient predictor. The adjacent nodes may be, for example, only the nodes adjacent to the surface, or may be the nodes adjacent to the surface and the nodes adjacent to the edge.

[0355] The RAHT unit 2080 may perform the same operation in the DC coefficient inter prediction in step S28003.

[0356] DC pred = αDC inter Here, let the DC coefficient of the reference node be DC inter and let the predicted value of the DC coefficient of the root node be DC pred .

[0357] Also, the RAHT unit 2080 may calculate the predicted value of the attribute value or the AC coefficient by combining the inter prediction and the intra prediction.

[0358] For example, an example of the case where the RAHT unit 2080 obtains the prediction of the attribute value is shown below.

[0359] Attr pred = W inter · Attr inter + W intra · Attr intra Here, Attr inter and Attr intra are the inter prediction and the intra prediction of the attribute value, respectively. Also, W inter and W intra are the weights of the inter prediction and the intra prediction, respectively.

[0360] W inter and W intra may be determined such that the intra prediction is more emphasized for deeper layers according to the depth depth of the processing target layer. For example, W inter = 1 - depth / N W intra = depth / N Let N be the maximum value of the depth of the layer where inter prediction is valid. The combination of inter prediction and intra prediction may be valid only at a specific layer. For example, the combination of inter prediction and intra prediction may be valid only when M < depth < N. M is any real number less than N and may be decoded as header information such as APS.

[0361] For example, the RAHT unit 2080 may perform bidirectional prediction. Hereinafter, an example of the operation of the RAHT unit 2080 when performing bidirectional prediction will be described. will be described.

[0362] First, the RAHT unit 2080 groups a certain number of frames to be processed, and processes them by swapping the processing order within the group.

[0363] For example, the RAHT unit 2080 may regard eight frames as one group, and process the frames from frame index 0 to frame index 7 within the group in the order of 0, 7, 1, 2, 3, 4, 5, 6.

[0364] Here, the in-group frame index is a number assigned for each order of the frames to be processed within the group.

[0365] Also, there may be two reference frames during inter prediction for each frame to be processed, and the frames to be referenced may be future frames in time series.

[0366] The in-group frame index order pattern and the frames referenced by the in-group frame index may be decoded as flags included in the APS 2611 or the ASH 2612.

[0367] Also, the RAHT unit 2080 may refer to the value of the above biPredictionPrediod and decode the in-group frame index order pattern and the frames referenced by the in-group frame index using such a value.

[0368] Here, the in-group frame index order pattern refers to the pattern of the order of in-group frame indexes.

[0369] Alternatively, in the raht_attr_layer_code_mode, for each layer, a flag indicating whether to perform intra prediction, inter prediction, no prediction, or bidirectional prediction may be included.

[0370] Also, when there are multiple in-group frame index order patterns in bidirectional prediction, the above raht_attr_layer_code_mode may include that number of variations.

[0371] Furthermore, the RAHT unit 2080 may prepare a list of reference frames and select, from within such a list, the frame referred to by each in-group frame index based on the value of the decoded in-list index.

[0372] Note that the RAHT unit 2080 may prepare two lists of frames that are past and future in time series as such a list of reference frames with respect to the frame to be processed, and may update them at the timing of processing each frame.

[0373] Also, the RAHT unit 2080 may fix and hard-code the frame referred to by each in-group frame index for each in-group frame index order pattern.

[0374] Also, the RAHT unit 2080 may decode the above raht_attr_layer_code_mode for each slice or for each layer.

[0375] (Point cloud encoding device 100) Hereinafter, with reference to FIG. 20, the point cloud encoding apparatus 100 according to the present embodiment will be described. FIG. 20 is a diagram showing an example of the functional blocks of the point cloud encoding apparatus 100 according to the present embodiment.

[0376] As shown in FIG. 20, the point cloud encoding apparatus 100 includes a coordinate conversion unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information encoding unit 1050, a geometric information reconstruction unit 1060, a color conversion unit 1070, an attribute transfer unit 1080, a RAHT unit 1090, a LoD calculation unit 1100, a lifting unit 1110, an attribute information quantization unit 1120, an attribute information encoding unit 1130, and a frame buffer 1140.

[0377] The coordinate conversion unit 1010 is configured to perform conversion processing from the three-dimensional coordinate system of the input point cloud to an arbitrary different coordinate system. The coordinate conversion may, for example, convert the x, y, and z coordinates of the input point cloud to arbitrary s, t, and u coordinates by rotating the input point cloud. Also, as one variation of the conversion, the coordinate system of the input point cloud may be used as it is.

[0378] The geometric information quantization unit 1020 is configured to perform quantization of the position information of the input point cloud after coordinate conversion and removal of points with overlapping coordinates. Note that when the quantization step size is 1, the position information of the input point cloud and the position information after quantization match. That is, when the quantization step size is 1, it is equivalent to the case where quantization is not performed.

[0379] The tree analysis unit 1030 is configured to generate an occupancy code indicating in which node of the encoding target space a point exists based on the tree structure described later, using the position information of the point cloud after quantization as input.

[0380] In this process, the tree analysis unit 1030 is configured to generate a tree structure by recursively dividing the encoding target space into rectangular parallelepipeds.

[0381] Here, when there are points within a certain rectangular parallelepiped, a tree structure can be generated by recursively executing the process of dividing such a rectangular parallelepiped into a plurality of rectangular parallelepipeds until the rectangular parallelepiped reaches a predetermined size. Each such rectangular parallelepiped is referred to as a node. Also, each rectangular parallelepiped generated by dividing a node is called a child node, and the occupancy code is a representation of whether a point is included in the child node, expressed as 0 or 1.

[0382] As described above, the tree analysis unit 1030 is configured to generate an occupancy code while recursively dividing nodes until they reach a predetermined size.

[0383] In this embodiment, a method called "Octree" that always recursively performs octree division with the above-described rectangular parallelepiped as a cube, and a method called "QtBt" that performs quadtree division and binary tree division in addition to octree division can be used.

[0384] Here, whether to use "QtBt" is transmitted to the point cloud decoding device 200 as control data.

[0385] Alternatively, it may be specified to use Predictive geometry coding using an arbitrary tree structure. In such a case, the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoding device 200 as control data.

[0386] For example, the control data of the tree structure may be configured to be decodable by the procedure described in FIGS. 5 to 14.

[0387] The approximate surface analysis unit 1040 is configured to generate approximate surface information using the tree information generated by the tree analysis unit 1030.

[0388] The approximate surface information is, for example, when decrypting the 3D point cloud data of an object, etc., in the case where the point cloud is densely distributed on the object surface, instead of decrypting each individual point cloud, it is the result of approximating the existence region of the point cloud with a small plane and expressing it.

[0389] Specifically, the approximate surface analysis unit 1040 may be configured to generate approximate surface information, for example, by a method called "Trisoup". Also, when decrypting a sparse point cloud obtained by Lidar or the like, this process can be omitted.

[0390] The geometric information encoding unit 1050 is configured to encode the syntax such as the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040 to generate a bit stream (geometric information bit stream). Here, the bit stream may include, for example, the syntax described in FIG. 4.

[0391] The encoding process is, for example, context-adaptive binary arithmetic encoding. Here, for example, the syntax includes control data (flags and parameters) for controlling the decryption process of the position information.

[0392] The geometric information reconstruction unit 1060 is configured to reconstruct the geometric information (coordinate system assumed by the encoding process, that is, the position information after coordinate transformation in the coordinate transformation unit 1010) of each point of the point cloud data to be encoded based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040.

[0393] The frame buffer 1140 is configured to input the geometric information reconstructed by the geometric information reconstruction unit 1060 and store it as a reference frame.

[0394] When performing inter-prediction of frames at different times in the tree analysis unit 1030, the stored reference frame is read from the frame buffer 1140 and used as a reference frame.

[0395] Here, for each frame, which reference frame at which time is used is determined based on, for example, the value of a cost function representing the coding efficiency, and information on the reference frame to be used may be transmitted to the point cloud decoding device 200 as control data.

[0396] The color conversion unit 1070 is configured to perform color conversion when the input attribute information is color information. The color conversion does not necessarily have to be executed, and whether to execute the color conversion process is encoded as part of the control data and transmitted to the point cloud decoding device 200.

[0397] The attribute transfer unit 1080 is configured to correct the attribute value so that the distortion of the attribute information is minimized based on the position information of the input point cloud, the position information of the point cloud after reconstruction in the geometric information reconstruction unit 1060, and the attribute information after color change in the color conversion unit 1070. As a specific correction method, for example, the method described in Non-Patent Document 1 can be applied.

[0398] The RAHT unit 1090 takes as input the attribute information after transfer by the attribute transfer unit 1080 and the geometric information generated by the geometric information reconstruction unit 1060, and is configured to generate residual information for each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform).

[0399] The information to be decoded is the direct current component (DC coefficient) and the alternating current component (AC coefficient) of the attribute information generated by using RAHT in the encoding process, and is converted into attribute information by using the inverse transform of RAHT in the decoding process.

[0400] As a specific process of RAHT, for example, the method described in the above Non-Patent Document 1 can be used.

[0401] The LoD calculation unit 1100 is configured to generate LoD (Level of Detail) using the geometric information generated by the geometric information reconstruction unit 1060 as an input.

[0402] LoD is information for defining a reference relationship (points to be referred to and points being referred to) for realizing predictive coding such as predicting the attribute information of a certain point from the attribute information of another point and encoding or decoding the prediction residual.

[0403] In other words, LoD is information that defines a hierarchical structure in which each point included in the geometric information is classified into a plurality of levels, and for points belonging to a lower level, the attribute is encoded or decoded using the attribute information of points belonging to a higher level.

[0404] As a specific method for determining LoD, for example, the method described in Non-Patent Document 1 above may be used.

[0405] The lifting unit 1110 is configured to generate residual information by lifting processing using the LoD generated by the LoD calculation unit 1100 and the attribute information after attribute transfer in the attribute transfer unit 1080.

[0406] As a specific process of lifting, for example, the method described in Non-Patent Document 1 above may be used.

[0407] The attribute information quantization unit 1120 is configured to quantize the residual information output from the RAHT unit 1090 or the lifting unit 1110. Here, when the quantization step size is 1, it is equivalent to the case where quantization is not performed.

[0408] The attribute information encoding unit 1130 is configured to perform an encoding process using the quantized residual information output from the attribute information quantization unit 1120 as a syntax, and generate a bit stream (attribute information bit stream) regarding the attribute information.

[0409] The symbolization process is, for example, context-adaptive binary arithmetic symbolization processing. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0410] The point cloud encoding device 100 is configured to perform an encoding process using the position information and attribute information of each point in the point cloud as inputs through the above processes, and output a geometric information bitstream and an attribute information bitstream.

[0411] According to this embodiment, by using the DC coefficient, it is determined whether to apply intra prediction to the AC coefficient. If it is determined in advance that the accuracy of intra prediction is high, intra prediction is performed. If it is determined that the accuracy of intra prediction is not high, by not performing prediction, the amount of code for the AC coefficient to be decoded is reduced, and the encoding efficiency is improved.

[0412] Also, according to this embodiment, by scaling the intra-predicted attribute value or the AC coefficient obtained by RAHT of the attribute value, the prediction accuracy is improved, the residual to be decoded is reduced, and the encoding efficiency is improved.

[0413] Also, the above-described point cloud encoding device 100 and point cloud decoding device 200 may be realized by a program that causes a computer to execute each function (each process).

[0414] In each of the above embodiments, the present invention has been described by taking the application to the point cloud encoding device 100 and the point cloud decoding device 200 as an example. However, the present invention is not limited to such examples, and can be similarly applied to a point cloud encoding / decoding system having each function of the point cloud encoding device 100 and the point cloud decoding device 200.

Industrial Applicability

[0415] Note that according to this embodiment, for example, in video communication, since an overall improvement in service quality can be realized, it becomes possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization, and foster innovation."

Explanation of Signs

[0416] 10…Point cloud processing system 100…Point cloud encoding device 1010…Coordinate conversion unit 1020…Geometric information quantization unit 1030…Tree analysis unit 1040…Approximate surface analysis unit 1050…Geometric information encoding unit 1060…Geometric information reconstruction unit 1070…Color conversion unit 1080…Attribute transfer unit 1090…RAHT unit 1100…LoD calculation unit 1110…Lifting unit 1120…Attribute information quantization unit 1130…Attribute information encoding unit 200…Point cloud decoding device 2010…Geometric information decoding unit 2020…Tree synthesis unit 2030…Approximate surface synthesis unit 2040…Geometric information reconstruction unit 2050…Inverse coordinate conversion unit 2060…Attribute information decoding unit 2070…Inverse quantization unit 2080…RAHT unit 2090…LoD calculation unit 2100…Inverse lifting unit 2110…Inverse color conversion unit

Claims

1. A point cloud decoding device, comprising an RAHT unit, wherein in the decoding process of attribute values using RAHT, the RAHT unit performs scaling on the intra-predicted attribute values or the AC coefficients obtained by performing RAHT on the attribute values, and is characterized by the point cloud decoding device.

2. The point cloud decoding device according to claim 1, wherein the RAHT unit performs scaling on the intra-predicted attribute values using a hard-coded scaling factor.

3. The RAHT unit, decodes a scaling factor, and performs scaling on the intra-predicted attribute values using the scaling factor, and is characterized by the point cloud decoding device according to claim 1.

4. A point cloud decoding device, comprising an RAHT unit, wherein in the decoding process of attribute values using RAHT, the RAHT unit determines the applicability of bidirectional prediction using the applicability mode of inter-prediction for each layer, and is characterized by the point cloud decoding device.

5. A point cloud decoding device, comprising an RAHT unit, wherein for each order of frames to be processed within a group, the RAHT unit selects a reference frame for bidirectional prediction from within the list based on the prepared list of reference frames and the value of the index in the decoded list, and is characterized by the point cloud decoding device.

6. A point cloud decoding device, comprising a tree synthesis unit, wherein in the prediction process of position information, for each order of frames to be processed within a group, the tree synthesis unit selects a reference frame for bidirectional prediction from within the list based on the prepared list of reference frames and the value of the index in the decoded list, and is characterized by the point cloud decoding device.

7. A point cloud decoding device, comprising a tree synthesis unit, wherein when performing bilateral prediction in the prediction process of position information, the tree synthesis unit creates one predictor from among the two selected frames based on the predictor index of each decoded reference frame, and is characterized by the point cloud decoding device.

8. The point cloud decoding device according to claim 7, wherein the tree synthesis unit uses the average value of the predictors of two reference frames as the predictor.

9. The tree synthesis unit according to claim 7, wherein in the point cloud decoding apparatus, a weighted average using a weight according to the distance between the reference frame and the frame to be processed is used as a predictor based on predictors of two reference frames.

10. A point cloud decoding method, comprising: In the decoding process of attribute values using RAHt, a step of scaling an intra-predicted attribute value or an AC coefficient obtained by performing RAHt on the attribute value.

11. A program for causing a computer to function as a point cloud decoding apparatus, The point cloud decoding apparatus includes a RAHt unit, The RAHt unit is characterized in that in the decoding process of attribute values using RAHt, scaling is performed on an intra-predicted attribute value or an AC coefficient obtained by performing RAHt on the attribute value.

Images