Decoding method, encoding method, decoding device, and encoding device

Abstract

This decoding method includes: calculating a prediction value of a first coefficient of a first node included in a first tree structure which is a target encoding unit, by using an inter-prediction system referring to a reference encoding unit (S201); acquiring a residual value of the first coefficient from a bitstream (S202); calculating the first coefficient on the basis of the prediction value and the residual value (S203); performing inverse hierarchical transform on the first coefficient (S204); and calculating an attribute value of a three-dimensional point included in the target encoding unit, on the basis of a result of the inverse hierarchical transform (S205). For example, the inverse hierarchical transform may be inverse transform of the hierarchical transform that is applied to two coefficients of two adjacent nodes in order to calculate a coefficient of an upper node positioned upper than the two nodes.

Description

Decoding method, encoding method, decoding device, and encoding device

[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device.

[0002] In the future, devices and services that utilize 3D data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, video distribution, etc. 3D data can be acquired in a variety of ways, including distance sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

[0003] One method of representing three-dimensional data is a representation called a point cloud, which represents the shape of a three-dimensional structure using a group of points in three-dimensional space. In a point cloud, the position and color of the points are stored. Point clouds are expected to become the mainstream method of representing three-dimensional data, but point clouds have a very large amount of data. Therefore, when storing or transmitting three-dimensional data, it is essential to compress the amount of data by encoding, just as with two-dimensional moving images (examples include MPEG-4 AVC or HEVC standardized by MPEG).

[0004] Furthermore, compression of point clouds is partially supported by a public library (Point Cloud Library) that performs processing related to point clouds.

[0005] Furthermore, a technique is known in which three-dimensional map data is used to search for and display facilities located around a vehicle (see, for example, Patent Document 1).

[0006] International Publication No. 2014 / 020663

[0007] In such encoding and decoding methods, it is desirable to be able to improve the encoding efficiency.

[0008] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve encoding efficiency.

[0009] A decoding method according to one aspect of the present disclosure calculates a predicted value of a first coefficient of a first node included in a first tree structure of a target coding unit using an inter-prediction method that references a reference coding unit, obtains a residual value of the first coefficient from the bitstream, calculates the first coefficient based on the predicted value and the residual value, performs an inverse hierarchical transform on the first coefficient, and calculates an attribute value of a three-dimensional point included in the target coding unit based on the result of the inverse hierarchical transform.

[0010] An encoding method according to one aspect of the present disclosure calculates a first coefficient of a first node included in a first tree structure of a target coding unit by hierarchically transforming attribute values ​​of three-dimensional points included in the target coding unit, calculates a predicted value of the first coefficient using an inter-prediction method that references a reference coding unit, calculates a residual value that is the difference between the first coefficient and the predicted value, and generates a bitstream including the residual value.

[0011] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve encoding efficiency.

[0012] FIG. 1 is a diagram for explaining encoding of attribute information using an RAHT according to Embodiment 1. FIG. 2 is a diagram showing an example of an RAHT target node according to Embodiment 1. FIG. 3 is a diagram showing a three-dimensional area in an octree structure corresponding to an RAHT target node according to Embodiment 1. FIG. 4 is a diagram showing an example of the configuration of an RAHT target node and a child node of the target node according to Embodiment 1. FIG. 5 is a diagram showing example input values ​​and output values ​​of the RAHT according to Embodiment 1. FIG. 6 is a flowchart showing an example of processing of the RAHT method according to Embodiment 1. FIG. 7 is a flowchart of processing for determining whether intra-RAHT prediction is performed according to Embodiment 1. FIG. 8 is a block diagram of an encoding device according to Embodiment 1. FIG. 9 is a block diagram of a decoding device according to Embodiment 1. FIG. 10 is a flowchart of inter-prediction processing according to Embodiment 1. FIG. 11 is a diagram schematically showing an overview of inter-prediction processing according to Embodiment 1. FIG. 12 is a diagram for explaining a method for determining whether layers and positions match according to Embodiment 1. FIG. 13 is a diagram showing example data of a reference frame stored in a memory according to Embodiment 1. FIG. 14 is a diagram showing an example of the syntax of an APS according to the first embodiment. FIG. 15 is a block diagram of an encoding device according to the second embodiment. FIG. 16 is a block diagram of a decoding device according to the second embodiment. FIG. 17 is a diagram showing an example of a position correction process for a reference point group according to the second embodiment. FIG. 18 is a diagram showing an example of a position shift of a node in an RAHT tree according to the second embodiment. FIG. 19 is a diagram showing a three-dimensional area corresponding to the RAHT tree according to the second embodiment. FIG. 20 is a diagram showing an example of the syntax of an APS according to the second embodiment. FIG. 21 is a flowchart of a decoding process according to the embodiment. FIG. 22 is a block diagram of a decoding device according to the embodiment. FIG. 23 is a flowchart of an encoding process according to the embodiment. FIG. 24 is a block diagram of an encoding device according to the second embodiment.

[0013] A three-dimensional data decoding method according to one aspect of the present disclosure calculates a predicted value of a first coefficient of a first node included in a first tree structure of a target coding unit using an inter-prediction method that references a reference coding unit, obtains a residual value of the first coefficient from a bitstream, calculates the first coefficient based on the predicted value and the residual value, performs an inverse hierarchical transform on the first coefficient, and calculates an attribute value of a three-dimensional point included in the target coding unit based on the result of the inverse hierarchical transform.

[0014] According to this, when there is correlation between coefficients between coding units, the accuracy of inter prediction can be improved, and therefore coding efficiency can be improved.

[0015] For example, the inverse hierarchical transform may be an inverse transform of a hierarchical transform applied to two coefficients of two adjacent nodes to calculate the coefficients of a higher-order node located above the adjacent nodes. In such a hierarchical transform, correlations tend to appear between coefficients in coding units. Therefore, prediction accuracy can be improved, thereby improving coding efficiency.

[0016] For example, in the calculation of the predicted value, a second node included in a second tree structure of the reference coding unit may be referenced, and a second position of the second node in the second tree structure may be the same as a first position of the first node in the first tree structure. This can improve prediction accuracy by referencing the second node at the same position as the first node, where there is a high possibility that coefficients between coding units are correlated. This can improve coding efficiency.

[0017] For example, in the calculation of the predicted value, a second node included in a second tree structure of the reference coding unit may be referenced, and a difference between a second position of the second node in the second tree structure and a first position of the first node in the first tree structure may be less than a predetermined threshold. This can improve prediction accuracy by referencing a second node located close to the first node, which is likely to have a correlation between coefficients between coding units. Therefore, coding efficiency can be improved.

[0018] For example, the first location and the second location may be represented by a Morton code, which can be used to easily search for a second node at the same location as or close to the first node.

[0019] For example, in the hierarchical transform, values ​​of low-frequency components and values ​​of high-frequency components are generated, the low-frequency components correspond to the first coefficients, and the values ​​of the high-frequency components do not need to be predicted. This can improve coding efficiency by using prediction for low-frequency components, which tend to have high prediction accuracy. Furthermore, not using prediction for high-frequency components can reduce the amount of processing.

[0020] For example, the decoding method may further include storing the first coefficient in a buffer memory for calculating a predicted value of a coefficient of another node, whereby the first coefficient stored in the buffer memory can be used to predict a coefficient of another node.

[0021] For example, the decoding method may further include obtaining a quantized value obtained by quantizing the first coefficient, and storing the quantized value in a buffer memory for calculating a predicted value of another node without performing inverse quantization of the quantized value. This reduces the amount of data stored in the memory.

[0022] For example, the reference may be performed in units of nodes, which allows prediction to be applied when the correlation is partially high even when the overall correlation of the coding unit is not high, thereby improving coding efficiency.

[0023] For example, in the calculation of the predicted value, a second node included in a second tree structure of the reference coding unit may be referenced, and the predicted value may be calculated when the number of coefficients of the first node is the same as the number of coefficients of the second node. This can prevent erroneous prediction.

[0024] For example, the predicted value may be calculated when at least one of the number of 3D points included in the target coding unit, the arrangement of the 3D points, the density of the 3D points, and the depth of the first node in the first tree structure satisfies a predetermined condition. In this way, when there is a low possibility of correlation between coding units, prediction is not performed, thereby reducing the amount of processing.

[0025] For example, if the condition is satisfied, the first coefficient may be stored in a buffer memory for calculation of predicted values ​​of coefficients of other nodes, thereby reducing the amount of data stored in the memory.

[0026] For example, in the calculation of the predicted value, motion compensation may be performed on the reference coding unit, and the predicted value may be calculated by referring to the reference coding unit after motion compensation. This can improve prediction accuracy when there is motion between coding units, thereby improving coding efficiency.

[0027] For example, the decoding method may further store the calculated attribute values ​​in a buffer memory for calculating predicted values ​​of coefficients of other nodes, whereby the attribute values ​​stored in the buffer memory can be used to predict coefficients of other nodes.

[0028] For example, in calculating the predicted value, an attribute value of the reference coding unit may be calculated by performing an inverse hierarchical transformation on a second coefficient of the reference coding unit, motion compensation may be performed on the attribute value of the reference coding unit, and the predicted value may be calculated by performing a hierarchical transformation on the attribute value of the reference coding unit after motion compensation.

[0029] For example, a three-dimensional data decoding method according to one aspect of the present disclosure calculates a predicted value of a first coefficient of a first node included in a target coding unit using an inter-prediction method that references a reference coding unit, obtains a residual value of the first coefficient from a bitstream, calculates the first coefficient based on the predicted value and the residual value, and performs an inverse transform process on the first coefficient to calculate an attribute value of a three-dimensional point included in the target coding unit, where the first coefficient is generated by a transform process that is executed by an encoding device and converts the attribute value into the first coefficient.

[0030] According to this, when there is correlation between coefficients between coding units, the accuracy of inter prediction can be improved, and therefore coding efficiency can be improved.

[0031] In addition, a three-dimensional data encoding method according to one aspect of the present disclosure calculates a first coefficient of a first node included in a first tree structure of a target coding unit by hierarchically transforming attribute values ​​of three-dimensional points included in the target coding unit, calculates a predicted value of the first coefficient using an inter-prediction method that references a reference coding unit, calculates a residual value that is the difference between the first coefficient and the predicted value, and generates a bitstream including the residual value.

[0032] According to this, when there is correlation between coefficients between coding units, the accuracy of inter prediction can be improved, and therefore coding efficiency can be improved.

[0033] In addition, a decoding device according to one aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to calculate a predicted value of a first coefficient of a first node included in a first tree structure of a target coding unit using an inter prediction method that references a reference coding unit, obtains a residual value of the first coefficient from the bitstream, calculates the first coefficient based on the predicted value and the residual value, performs an inverse hierarchical transform on the first coefficient, and calculates an attribute value of a three-dimensional point included in the target coding unit based on the result of the inverse hierarchical transform.

[0034] In addition, an encoding device according to one aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to hierarchically transform attribute values ​​of three-dimensional points included in a target coding unit to calculate a first coefficient of a first node included in a first tree structure of the target coding unit, calculate a predicted value of the first coefficient using an inter-prediction method that references a reference coding unit, calculate a residual value that is the difference between the first coefficient and the predicted value, and generate a bitstream including the residual value.

[0035] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0036] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0037] (Embodiment 1) [RAHT Method] Hereinafter, a method using RAHT (Region Adaptive Hierarchical Transform) will be described as a method for encoding attribute information (also referred to as attribute values) of three-dimensional points included in a point cloud. Fig. 1 is a diagram for explaining encoding of attribute information using RAHT.

[0038] First, the encoding device (three-dimensional data encoding device) generates a Morton code based on the position information of the three-dimensional points, and sorts the attribute information of the three-dimensional points in order of the Morton code.

[0039] Next, the encoding device applies a Haar transform to the attribute information of two adjacent 3D points in Morton code order to generate high-frequency components and low-frequency components of layer L. For example, the encoding device may use a 2×2 matrix Haar transform. The generated high-frequency components are included in the encoding coefficients as high-frequency components of layer L, and the generated low-frequency components are used as input values ​​for layer L+1, which is higher than layer L.

[0040] After generating high-frequency and low-frequency components for layer L using the attribute information for layer L, the encoding device continues processing for layer L+1. In the processing for layer L+1, the encoding device generates high-frequency and low-frequency components for layer L+1 by applying a Haar transform to two low-frequency components obtained by Haar transforming the attribute information for layer L. The generated high-frequency components are included in the coding coefficients as high-frequency components for layer L+1, and the generated low-frequency components are used as input values ​​for layer L+2, which is higher than layer L+1.

[0041] The encoding device repeats this hierarchical processing until it determines that it has reached the highest hierarchical level Lmax when only one low-frequency component is input to the hierarchical level. The encoding device includes the low-frequency component of hierarchical level Lmax-1 input to hierarchical level Lmax in the encoding coefficients. The encoding device then quantizes the values ​​of the low-frequency or high-frequency components included in the encoding coefficients, and encodes the quantized values ​​using entropy coding or the like.

[0042] In addition, when only one three-dimensional point exists as two adjacent three-dimensional points when applying the Haar transform, the encoding device may use the value of the attribute information of the one existing three-dimensional point as the input value of the higher hierarchy.

[0043] In this way, the encoding device applies a hierarchical Haar transform to the input attribute information to generate high-frequency and low-frequency components of the attribute information, and then performs encoding by applying quantization, etc., which will be described later, thereby improving encoding efficiency.

[0044] When the attribute information is N-dimensional, the encoding device may apply the Haar transform to each dimension independently to calculate the respective encoding coefficients. For example, when the attribute information is color information (RGB, YUV, etc.), the encoding device applies the Haar transform to each component to calculate the respective encoding coefficients.

[0045] The encoding device may apply the Haar transform in the order of layer L, L+1, ..., layer Lmax. The closer to layer Lmax, the more encoding coefficients are generated that include low-frequency components of the input attribute information.

[0046] In the example shown in Fig. 1, the input attribute information is a0, a1, a2, a3, a4, and a5. Of the coding coefficients after Haar transform, Ta1, Ta5, Tb1, Tb3, Tc1, and d0 are coded. The other coding coefficients (b0, b2, c0, etc.) are intermediate values ​​and are not coded.

[0047] 1, a high-frequency component Ta1 and a low-frequency component b0 are generated by performing a Haar transform on a0 and a1. Here, when weights w0 and w1 (described later) are equal, the low-frequency component b0 is the average value of a0 and a1, and the high-frequency component Ta1 is the difference between a0 and a1.

[0048] Since there is no attribute information paired with a2, a2 ​​is used as b1 as is. Similarly, there is no attribute information paired with a3, so a3 is used as b2 as is. Furthermore, by performing Haar transform on a4 and a5, a high-frequency component Ta5 and a low-frequency component b3 are generated. For example, b1, b2, b3, and Ta5 are expressed by the following equations 1 to 4.

[0049]

[0050] w0 and w1 are weights assigned to each 3D point. For example, the encoding device may calculate the weights based on distance information between two adjacent 3D points to which the Haar transform is applied. For example, the encoding device may improve encoding efficiency by increasing the weight as the distance becomes shorter. Note that the encoding device may calculate these weights using a different method, or may not use weights at all. In other words, w0 = w1 may be true.

[0051] At layer L+1, a Haar transform is performed on b0 and b1 to generate a high-frequency component Tb1 and a low-frequency component c0. Similarly, a Haar transform is performed on b2 and b3 to generate a high-frequency component Tb3 and a low-frequency component c1. For example, Tb1, c0, Tb3, and c1 are expressed by the following equations 5 and 6.

[0052]

[0053] At layer Lmax-1, a high frequency component Tc1 and a low frequency component d0 are generated by performing a Haar transform on c0 and c1. For example, Tc1 and d0 are expressed by the following equation 7.

[0054]

[0055] The encoding device may quantize the coding coefficients after applying the Haar transform and then encode them. For example, the encoding device performs quantization by dividing the coding coefficients by a quantization scale (also called a quantization step (QS)). In this case, the smaller the quantization scale, the smaller the error (quantization error) that may occur due to quantization. Conversely, the larger the quantization scale, the larger the quantization error.

[0056] Next, the inverse Haar transform will be described. A decoding device (three-dimensional data decoding device) decodes attribute values ​​of three-dimensional points by applying the inverse Haar transform to the coding coefficients after dequantization.

[0057] First, the decoding device generates a Morton code based on the position information of the three-dimensional points and sorts the three-dimensional points in Morton code order. Next, the decoding device applies an inverse Haar transform to the coding coefficients including low-frequency components of layer L+1 and the coding coefficients including high-frequency components of layer L, thereby restoring attribute information of adjacent three-dimensional regions (nodes) in Morton code order in layer L (adjacent three-dimensional points in the lowest layer). For example, the decoding device may use an inverse Haar transform of a 2×2 matrix. The restored attribute information of layer L is used as input values ​​for the lower layer L−1.

[0058] The decoding device repeats this hierarchical processing and terminates the processing when all the attribute information in the lowest layer has been decoded. Note that, when applying the inverse Haar transform, if only one three-dimensional area exists as two adjacent three-dimensional areas in layer L-1, the decoding device may substitute the value of the coded component of layer L for the attribute value of the one existing three-dimensional area. This allows the decoding device to apply the Haar transform to all values ​​of the input attribute information and correctly decode a bitstream with improved coding efficiency.

[0059] The decoding device may apply the inverse Haar transform in the order of layers Lmax, Lmax-1, ..., layer L. In the example shown in Figure 1, the coding coefficients after inverse quantization are Ta1, Ta5, Tb1, Tb3, Tc1, and d0, and the decoded values ​​obtained are a0, a1, a2, a3, a4, and a5.

[0060] For example, c0 and c1 obtained by the inverse Haar transform of the layer Lmax-1 are expressed by the following equation 8. Note that a and b are expressed by the above equation 4.

[0061]

[0062] For example, b0, b1, b2, and b3 obtained by the inverse Haar transform of the layer L+1 are expressed by the following equations 9 and 10.

[0063]

[0064] For example, a0, a1, a2, a3, a4, and a5 obtained by the inverse Haar transform of layer L are expressed by the following equations 11 to 14.

[0065]

[0066] [Octree Node and RAHT Method] Fig. 2 is a diagram showing an example of a RAHT target node. Fig. 3 is a diagram showing a three-dimensional area in an octree structure corresponding to the RAHT target node shown in Fig. 2.

[0067] In the RAHT method, for example, frequency conversion processing (see FIG. 1) is performed in units of 2x2x2 voxels. FIG. 3 shows an octree representation of the position information of point clouds. Processing is performed for each node included in the octree hierarchy shown in FIG.

[0068] For example, let us consider a case where a first node, which is a RAHT target node located in octree layer N, is used as an input value. In this case, the lowest frequency component among eight frequency components corresponding to the three-dimensional regions of up to eight child nodes of the first node located in octree layer N+1 is used as an input value, and up to eight transform coefficients (frequency components) corresponding to the three-dimensional region of the first node are output by performing RAHT. The lowest frequency component among the eight output transform coefficients is used as one of the input values ​​in the RAHT of the second node, which is the parent node of the first node. Note that if octree layer N+1 is the lowest layer in the octree layer, the three-dimensional region of the node in that layer is a point, and the value is an attribute value of the point. In other words, in the RAHT transformation of the first node, the attribute values ​​of up to eight child nodes (points) of the first node are used as input values.

[0069] Here, the hierarchical layers of the octree are defined as depth, and are numbered from the top layer as layer 0, layer 1, layer 2, etc. Note that the output values ​​obtained by RAHT are called transform coefficients, but may also be called coding coefficients, RAHT transform values, or Haar transform values.

[0070] [Dyadic RAHT Method] Next, RAHT processing by the Dyadic RAHT method for each node in the octree hierarchy will be described. Fig. 4 is a diagram showing an example of the configuration of a target node of RAHT and its child nodes. Fig. 5 is a diagram showing an example of input and output values ​​of RAHT.

[0071] In the RAHT of a target node included in octree layer N, attribute values ​​(if octree layer N+1 is the lowest layer) or frequency components (if octree layer N+1 is not the lowest layer) corresponding to the child node (octree layer N+1) of the target node are input. Next, for each pair of two subnodes adjacent to each other in either axis direction, a predetermined transformation formula (e.g., Haar transformation) is used to output high-frequency components and low-frequency components. Note that subnodes are also called nodes or child nodes, but since these subnodes are obtained by dividing octree nodes, they are referred to as subnodes here to distinguish them from octree nodes.

[0072] As shown in Figure 5, the output value obtained by the conversion process for one axis constitutes a subnode on layer L1. Next, a similar conversion process is performed on the second and third axes to derive a subnode on layer L3. This subnode on layer L3 is the output value of the RAHT. Note that layers L1 to L3 here are layers in the RAHT and are represented as a binary tree.

[0073] Here, the input value has a value when a point exists in the child node, so it can also be used as information on whether or not the child node contains a point. There are a maximum of eight child nodes, and if points exist in all child nodes, there are a maximum of eight input values. Also, Coeff, which is the conversion coefficient (output value) after RAHT, ｉｎ（ｉ） Coeff with a value other than 0 (i is 0 to 7) ｉｎ（ｉ） The number of nodes varies depending on whether or not there are points for child nodes, and is a maximum of eight.

[0074] The input value of RAHT is the lowest frequency component of the transform coefficients of the child nodes of the target node. For example, the input value Coeff shown in FIG. ｃｈｉｌｄ０（０） is the lowest frequency component among the eight transform coefficients calculated for the child node with child node number 0. If the child node belongs to the lowest layer, the input value is the attribute value of the point included in the child node.

[0075] The frequency values ​​of two adjacent nodes are converted into low-frequency and high-frequency components in three stages for each axis, and the eight frequency components Coeff corresponding to the target node are calculated. ｉｎ（０） ~Coeff ｉｎ（７） is output. Here, Coeff ｉｎ（０） is the lowest frequency component, i.e., the DC component, and is used as the input value of the RAHT of the parent node. ｉｎ（７） is the highest frequency component. ｉｎ（ｉ） (i is 0 to 7) The larger the value of i, the higher the frequency component.

[0076] On the other hand, Coeff ｉｎ（１） ~Coeff ｉｎ（７） is quantized, and the quantized Coeffｉｎ（１） ~Coeff ｉｎ（７） is entropy coded (arithmetic coded). ｉｎ（０） is also quantized, and the quantized Coeff ｉｎ（０） is also entropy coded.

[0077] That is, in the Dyadic RAHT method, hierarchical RAHT is also performed on frequency components other than the highest frequency component that was quantized and coded as is in the process shown in FIG.

[0078] [Intra prediction in RAHT method] RAHT is performed for each node in the octree hierarchy. The order of the nodes is determined for each hierarchy to which the node belongs, for example, in Morton code order. Note that the hierarchy or Morton code order may be ascending or descending.

[0079] 6 is a flowchart showing an example of processing in the RAHT method. The processing shown in FIG. 6 is repeatedly performed for each node.

[0080] First, the encoding device determines whether to perform intra-RAHT prediction by determining whether the conditions for performing intra-RAHT prediction are satisfied (S101). If the conditions for performing intra-RAHT prediction are satisfied (Yes in S101), the encoding device performs intra-RAHT prediction and encodes information about the target node (S102). On the other hand, if the conditions for performing intra-RAHT prediction are not satisfied (No in S101), the encoding device does not perform intra-RAHT prediction and encodes information about the target node without using prediction (S103).

[0081] Here, intra-RAHT prediction is a prediction process that uses information on other nodes included in a target frame that includes a target node. For example, in intra-RAHT prediction, an encoding device calculates a predicted value from attribute values ​​of neighboring nodes near the target node. Next, the encoding device calculates predicted transform coefficients by performing RAHT on the predicted value. Next, the encoding device calculates a prediction residual, which is the difference between the transform coefficients obtained by performing RAHT on the target node and the predicted transform coefficients. Next, the encoding device performs arithmetic coding (entropy coding) on ​​the predicted residual.

[0082] In step S101, the encoding device determines whether to perform intra-RAHT prediction using, for example, node density. Specifically, if the density is high, the encoding device determines to perform intra-RAHT prediction, and if the density is low, the encoding device determines not to perform intra-RAHT prediction. For example, the encoding device determines the density using the number of neighboring nodes of a grandparent node and a parent node. Figure 7 is a flowchart of this determination process (S101).

[0083] First, the encoding device determines whether the number of neighboring nodes of the grandparent node of the target node is equal to or greater than a first threshold (S111). Note that a neighboring node is a node that includes one or more points located near the target node (the grandparent node in the above example) (e.g., whose distance from the target node is less than a predetermined value).

[0084] If the number of neighboring nodes of the grandparent node is equal to or greater than a first threshold (Yes in S111), the encoding device determines whether the number of neighboring nodes of the parent node is equal to or greater than a second threshold (S112).If the number of neighboring nodes of the parent node is equal to or greater than the second threshold (Yes in S112), the encoding device determines that the conditions for performing intra-RAHT prediction are satisfied (S113).

[0085] On the other hand, if the number of neighboring nodes of the grandparent node is less than the first threshold (No in S111), or if the number of neighboring nodes of the parent node is less than the second threshold (No in S112), the encoding device determines that the conditions for performing intra-RAHT prediction are not met (S114).

[0086] Although an example of the processing of the encoding device has been described here, the same processing may also be performed in the decoding device. Furthermore, the same processing may also be performed in the decoding device for the processing of the encoding device described later. Furthermore, the encoding device may perform step S113 when at least one of the conditions of step S111 and step S112 is satisfied.

[0087] [Inter Prediction in RAHT Method] The inter prediction method (temporal prediction method) in encoding attribute values ​​using RAHT will be described.

[0088] Currently, there is no technology that improves coding efficiency in encoding and decoding attribute values ​​of point clouds using the RAHT method by utilizing attribute values ​​of point clouds at different times. Point clouds at different times have different point locations or the presence or absence of points in a given region, and also have different attribute values.

[0089] On the other hand, even for point clouds acquired at different times, if certain conditions are met, the correlation between the attribute values ​​or conversion coefficients of nearby points may be high. For example, the correlation between conversion coefficients in point clouds acquired at different times varies depending on the number, arrangement, density, or node hierarchy of the point clouds. Furthermore, the correlation tends to be high when the acquisition range of the sensor acquiring the point clouds is the same, when the movement speed of the sensor is low, etc.

[0090] The method described in this embodiment is an inter-prediction method. In this method, when encoding a target frame, which is a point cloud frame to be processed, an encoding device calculates a predicted value by predicting transform coefficients using information included in a reference frame, which is a point cloud frame at a different time from the target frame, calculates a difference value (also called a prediction residual or residual value) between the transform coefficients of the target frame and the predicted value, and encodes the difference value. For example, the transform coefficients used to calculate the predicted value are transform coefficients included in a reference frame at a different time from the target frame and corresponding to the same spatial position and the same spatial size as the transform coefficients of the target frame. This allows for highly correlated predictions.

[0091] In addition, the encoding device may use prediction when the correlation between transform coefficients is high, and not use prediction when the correlation between transform coefficients is not high. In this way, the encoding device may have a function to determine whether to use prediction. This makes it possible to improve the coding efficiency of attribute values ​​using RAHT.

[0092] The method described in this embodiment can be applied to predicting a transformation value (transformation coefficient) obtained by a transformation process on an attribute value other than RAHT.

[0093] Although an example will be described here in which the point cloud used for prediction (reference point cloud) is included in a frame that is different in time from the target frame, the point cloud used for prediction may be a point cloud (reference point cloud) different from the target point cloud included in the target frame. For example, each of the target point cloud and the reference point cloud may be a point cloud included in a data unit (e.g., a tile or a slice) obtained by dividing the target frame.

[0094] The frame at a different time may be a frame immediately before the target frame or a frame immediately after the target frame. Prediction may also be performed using multiple frames.

[0095] Alternatively, the reference frame may be a frame that is different from the target frame but is the same in time as the target frame. For example, the target frame and the reference frame may be a left-eye frame and a right-eye frame that are the same in time.

[0096] [Configuration of Encoding Device and Decoding Device] The encoding device according to this embodiment includes a memory that stores transform coefficients of multiple decoded 3D points, and uses the transform coefficients to predict 3D points that have not yet been coded. For example, the transform coefficients are used to predict transform coefficients of attribute values ​​of 3D points included in a slice or frame different from the slice or frame to which the target 3D point belongs. The different slice or frame is, for example, a slice or frame that is different in time from the slice or frame to which the target 3D point belongs. Note that the memory may store values ​​obtained by inverse quantizing or inverse Haar transforming the transform coefficients.

[0097] Here, the conversion coefficient of the three-dimensional point is, for example, the frequency component Coeff in the Dyadic RAHT method shown in FIG. ｉｎ（０） ~Coeff ｉｎ（７） is.

[0098] 8 is a block diagram showing the configuration of an encoding device 100 according to this embodiment. Note that, although only a processing unit related to encoding attribute values ​​of a plurality of 3D points included in point cloud data is described here, the encoding device 100 may also include other processing units, such as a processing unit that encodes position information of a plurality of 3D points. For example, the encoding device 100 may encode the position information using an octree.

[0099] Specifically, in an octree, the target space is divided into eight nodes (subspaces), and 8-bit information (occupancy code) indicating whether a point cloud is included in each node is generated. The node including the point cloud is further divided into eight nodes, and 8-bit information indicating whether a point cloud is included in each of the eight nodes is generated. This process is repeated until the number of point clouds included in a predetermined layer or node falls below a threshold.

[0100] The encoding device 100 also has a function of performing intra prediction and inter prediction, and may switch between using intra prediction, inter prediction, or no prediction depending on a predetermined condition.

[0101] As shown in FIG. 8 , the encoding device 100 includes a transform unit 101, a subtraction unit 102, a switching unit 103, a quantization unit 104, an arithmetic encoding unit 105, an inverse quantization unit 106, and a first inter prediction unit 107.

[0102] The transform unit 101 generates transform coefficients by applying, for example, a Haar transform to the attribute information. The Haar transform includes a process of weighting the attribute values, and the attribute values ​​may be calculated using the weights. For example, the weights are generated based on information about the presence or absence of points or nodes in the RAHT tree.

[0103] The subtraction unit 102 calculates a difference value (also referred to as a prediction residual or a residual value) which is the difference between the transform coefficient generated by the transform unit 101 and the first predicted value generated by the first inter prediction unit 107. The switching unit 103 receives as input the transform coefficient output from the transform unit 101 and the difference value output from the subtraction unit 102. The switching unit 103 determines whether or not to use inter prediction, and outputs the difference value to the quantization unit 104 if inter prediction is used, or the transform coefficient if inter prediction is not used.

[0104] The quantization unit 104 generates quantized coefficients by quantizing the transform coefficients or the difference values. The arithmetic coding unit 105 generates a bitstream (coded data) by arithmetically coding the quantized coefficients. Note that the quantized coefficients may be binarized before arithmetic coding. The coding device 100 may also generate various header information and store the generated header information in the bitstream. Alternatively, this header information may be arithmetically coded and then stored in the bitstream. The inverse quantization unit 106 restores the transform coefficients or the difference values ​​by inverse quantizing the quantized coefficients.

[0105] The first inter prediction unit 107 includes a switching unit 111, an adder 112, and a frame buffer 113. When inter prediction is used, the switching unit 111 outputs the difference value obtained by the inverse quantization unit 106 to the adder 112, and when inter prediction is not used, the switching unit 111 outputs the transform coefficient obtained by the inverse quantization unit 106 to the frame buffer 113. The adder 112 generates a transform coefficient by adding the difference value and the first predicted value, and outputs the generated transform coefficient to the frame buffer 113. The first predicted value is read from the frame buffer 113.

[0106] The frame buffer 113 is a memory that stores transform coefficients of coded and decoded three-dimensional points, and stores the transform coefficients output from the switching unit 111 or the addition unit 112. For example, the frame buffer 113 stores transform coefficients for one frame or one data unit. Note that the frame buffer 113 may also store transform coefficients for multiple frames or multiple data units. The first inter prediction unit 107 generates a first predicted value using the transform coefficients stored in the frame buffer 113. For example, the transform coefficients stored in the frame buffer 113 are output as the first predicted value.

[0107] Fig. 9 is a block diagram showing the configuration of a decoding device 200 according to this embodiment. This decoding device 200 generates attribute values ​​of a plurality of 3D points included in point cloud data by decoding, for example, the bit stream generated by the encoding device 100 shown in Fig. 8.

[0108] Although only a processing unit related to decoding attribute values ​​of multiple 3D points included in point cloud data is described here, the decoding device 200 may also include other processing units, such as a processing unit that decodes position information of multiple 3D points. For example, the decoding device 200 may decode the position information using an octree. Furthermore, the decoding device 200 has a function of performing intra prediction and inter prediction, and may switch between using intra prediction, inter prediction, or no prediction depending on predetermined conditions.

[0109] As shown in FIG. 9 , the decoding device 200 includes an arithmetic decoding unit 201 , an inverse quantization unit 202 , a switching unit 203 , an addition unit 204 , an inverse transform unit 205 , and a first inter prediction unit 206 .

[0110] The arithmetic decoding unit 201 generates quantized coefficients by arithmetically decoding the bitstream. Note that the arithmetic decoding unit 201 may decode various types of header information from the bitstream.

[0111] The inverse quantization unit 202 generates transform coefficients or difference values ​​by inverse quantizing the quantized coefficients obtained by the arithmetic decoding unit 201. The switching unit 203 determines whether or not to use inter prediction, and if inter prediction is used, outputs the difference values ​​obtained by the inverse quantization unit 202 to the addition unit 204, and if inter prediction is not used, outputs the transform coefficients obtained by the inverse quantization unit 202 to the inverse transform unit 205. Note that whether inter prediction or intra prediction should be used is determined according to control information included in the bitstream, for example.

[0112] The adder 204 generates transform coefficients by adding the difference values ​​and the first predicted values ​​generated by the first inter predictor 206. The inverse transformer 205 generates attribute values ​​by applying an inverse Haar transform to the transform coefficients output from the switching unit 203 or the adder 204. The inverse Haar transform may also include a process of undoing the weighting of the attribute values ​​performed in the Haar transform.

[0113] The first inter prediction unit 206 includes a frame buffer 207. The frame buffer 207 is a memory that stores transform coefficients of decoded three-dimensional points. For example, the frame buffer 207 stores transform coefficients for one frame or one data unit. Note that the frame buffer 207 may store transform coefficients for multiple frames or multiple data units. The first inter prediction unit 206 generates a first predicted value using the transform coefficients stored in the frame buffer 207. For example, the transform coefficients stored in the frame buffer 207 are output as the first predicted value.

[0114] [Inter-Prediction Processing of Transform Coefficients] Fig. 10 is a flowchart of the inter-prediction processing in the encoding device 100. The processing shown in Fig. 10 is performed repeatedly for each node in the octree hierarchy included in the target frame, for example.

[0115] First, the encoding device 100 performs RAHT on the target node (S121). Next, the encoding device 100 determines whether the depth (level) to which the target node belongs is included in a predetermined target depth for inter prediction (S122). If the depth to which the target node belongs is included in the target depth for inter prediction (Yes in S122), the encoding device 100 determines to apply inter prediction to the target node, and if the depth to which the target node belongs is not included in the target depth for inter prediction (No in S122), the encoding device 100 determines not to apply inter prediction to the target node.

[0116] For example, nodes at shallower depths (higher levels) have transform coefficients corresponding to a wider space, so there is a high possibility that the correlation between the transform coefficients is high even between frames at different times. Thus, the possibility of high correlation varies depending on the depth. Therefore, there is a possibility that coding performance can be improved by switching whether or not to apply inter-prediction depending on the depth.

[0117] Furthermore, the encoding device 100 may determine a depth (a depth range, an upper limit, or a lower limit) of a target for inter prediction and store depth information indicating the determined depth in a bitstream. For example, the depth information may be stored in a Sequence Parameter Set (SPS), an Attribute Parameter Set (APS), or an Attribute Data Unit (ADU) header.

[0118] Note that SPS is metadata (parameter set) common to multiple frames. APS is metadata (parameter set) related to encoding of attribute information (attribute values). For example, APS is metadata common to multiple frames. ADU is a data unit of encoded data of attribute information. ADU header is the header (control information) of the ADU.

[0119] Furthermore, the encoding device 100 may determine the depth of the inter prediction target in advance, or may determine the depth adaptively according to the characteristics of the point cloud. The depth may be the same for the entire sequence, or a different depth may be set for each frame or each divided data.

[0120] Note that a node belonging to the highest layer is always defined as a node to be subjected to inter prediction, and the depth information may indicate the depth of the node to be subjected to inter prediction excluding the highest layer. For example, the depth information may be information indicating the number of depths in ascending order of depth as information indicating the depth (depth range). Note that the format of the information indicating the depth (depth range) is not limited to this. For example, the information may indicate the number of depths in descending order of depth.

[0121] If the depth to which the target node belongs is included in the depth to be subjected to inter prediction (Yes in S122), the encoding device 100 then determines whether or not a node at the same position as the target node exists in the reference frame (reference point group) (S123). If a node at the same position as the target node exists (Yes in S123), the encoding device 100 determines to apply inter prediction, and if no node at the same position as the target node exists (No in S123), the encoding device 100 determines not to apply inter prediction.

[0122] This is because the configurations of the point clouds in the target frame and the reference frame are different, and the reference frame may not have a node at the same position as the target node.

[0123] When the encoding device 100 determines to apply inter prediction (Yes in S122 and Yes in S123), it applies inter prediction to the target node and encodes the difference value between the transform coefficient and the predicted value (S124). On the other hand, when the encoding device 100 determines not to apply inter prediction (No in S122 or No in S123), it does not apply inter prediction to the target node and encodes the transform coefficient (S125).

[0124] After step S124 or S125, the encoding device 100 stores the derived transform coefficients in memory (frame buffer 113) (S126).

[0125] Although the processing of the encoding device 100 has been described above, the same processing may also be performed in the decoding device 200. In this case, "encoding" in the above description may be replaced with "decoding."

[0126] 11 is a diagram illustrating an outline of inter prediction processing. When it is determined that inter prediction is to be applied to a target node, the encoding device 100 performs the process of ｉｎ（０） ~Coeff ｉｎ（７） and a difference value between the transform coefficient of a reference node corresponding to the target node and included in a reference frame stored in the frame buffer 113, and the difference value is entropy coded (arithmetic coded). Note that the reference node is a node referenced by the target node in inter prediction, and is, for example, a node at the same position as the target node.

[0127] Note that the context used for entropy coding may be different between a node that applies inter prediction and a node that does not apply inter prediction. For example, a first context may be commonly used for multiple nodes that apply inter prediction, and a second context different from the first context may be commonly used for multiple nodes that do not apply inter prediction. Since there is a high correlation between the value characteristics of the target node and the reference node in nodes that apply inter prediction, using the same context may improve coding performance.

[0128] When the correlation between the transform coefficients of the target node and the transform coefficients of the reference node is high, the difference value becomes small, and encoding a small difference value may improve encoding efficiency. Note that the input value of the RAHT corresponds to the DC component of the RAHT of the child node group. Furthermore, the output value of the RAHT corresponds to the transform coefficient of the target node or the frequency component of the target node. Therefore, the transform coefficient of the target node can also be expressed as the transform coefficient of the frequency component of the target node or the DC component of the RAHT of the child node group.

[0129] Although FIG. 11 illustrates an example in which all nodes exist, some nodes may not exist depending on the configuration of the point cloud.

[0130] Next, an example of a method for determining whether or not a node at the same position as the target node exists in the reference frame in step S123 of FIG. 10 will be described.

[0131] For example, the RAHT output value of a node included in a reference frame and the position information (or Morton code) of the node are stored in a memory (e.g., the frame buffer 113). In this case, the encoding device 100 searches the data stored in the memory and determines whether a node having the same position information as the position information (or Morton code) of a target node included in the target frame exists in the reference frame. This allows the encoding device 100 to determine whether a node at the same position as the target node exists in the reference frame. Alternatively, a memory location (address within memory) for storing each piece of position information may be predetermined. In this case, the encoding device 100 may determine whether a node at the same position as the target node exists by determining whether data exists in a memory location corresponding to the position information of the target node.

[0132] For example, the encoding device 100 determines whether or not the blocks exist at the same position based on whether or not the layers and positions match. Fig. 12 is a diagram for explaining this determination method.

[0133] For example, a Morton code may be used to determine whether the layer and position match or not. The Morton code is a method for representing three-dimensional space one-dimensionally, with every three bits indicating the three-dimensional positional relationship of one layer. Therefore, if the layer is depth and the number of layers in the node tree is depthNum, the position pos can be expressed using mortonCode as follows:

[0134] pos=mortonCode>>((depthNum-depth)×3)

[0135] Here, matching of layers means that the size of the space corresponding to the target node is the same as the size of the space corresponding to the node included in the reference frame, and matching of Morton codes at the same layer means that the size of the space is the same and the spatial position is also the same.

[0136] In other words, when predicting a transform coefficient for a specific spatial position and a specific spatial size including a point group in a target frame, the encoding device 100 searches for a transform coefficient for the same spatial position and the same spatial size in a reference frame, and uses the searched transform coefficient as a predicted value. This enables prediction from transform coefficients with high correlation, thereby improving encoding efficiency.

[0137] Next, a method for determining whether to apply inter-prediction to each transform coefficient will be described. Whether or not the target node has a child node may also vary depending on whether the target node has eight transform coefficients. Therefore, even if information about a node at the same position as the target node exists in memory (e.g., frame buffer 113), the node may not have a transform coefficient corresponding to the target node.

[0138] Therefore, the encoding device 100 may determine whether or not there is a transform coefficient corresponding to the target node for each transform coefficient, and calculate a difference value if there is a corresponding transform coefficient. Note that if there is no transform coefficient for a node in the reference frame, the encoding device 100 may store a value of 0 in memory as the transform coefficient, and calculate a difference value regardless of whether there is a transform coefficient corresponding to the node in the reference frame.

[0139] In addition, inter-prediction may be applied if the presence or absence of all transform coefficients in the target node matches between the target frame and the reference frame, and inter-prediction may not be applied if the presence or absence of at least some transform coefficients does not match.

[0140] Alternatively, the encoding device 100 may apply inter prediction to only some of the transform coefficients of the target node, and not to the remaining transform coefficients. For example, some of the transform coefficients to which prediction is applied may be coefficients of predetermined frequency components. Alternatively, the encoding device 100 may adaptively switch which frequency components to which prediction is applied using a predetermined method.

[0141] For example, the application of prediction to low-frequency components tends to improve the coding efficiency. Therefore, the coding device 100 may apply prediction to low-frequency components but not to high-frequency components. This may maximize the effect while reducing the amount of data stored in the buffer and the amount of processing. However, in some cases, the coding device 100 may apply prediction to high-frequency components but not to low-frequency components.

[0142] Furthermore, if there is no node or transform coefficient corresponding to the target node in the reference frame, the encoding device 100 may use the transform coefficient of an adjacent node adjacent to the target node for inter prediction. Alternatively, the encoding device 100 may generate a predicted value by performing processing such as interpolation, interpolation, or averaging on the transform coefficients of multiple adjacent nodes.

[0143] Next, a description will be given of data buffered in a memory (for example, the frame buffer 113). The data stored in the memory may be transform coefficients after RAHT, transform coefficients after quantization and inverse quantization of the transform coefficients, transform coefficients after quantization but before inverse quantization (quantized coefficients), or attribute values ​​after inverse RAHT is applied to the transform coefficients.

[0144] If the attribute values ​​after performing inverse RAHT are saved, RAHT will need to be performed again in inter-prediction, increasing the amount of processing, but there is the advantage that the attribute values ​​before RAHT can be used in other processes.

[0145] On the other hand, when the transform coefficients are stored, RAHT is not required in inter prediction, which has the effect of reducing the amount of processing.

[0146] Furthermore, by storing the transform coefficients (quantized coefficients) after quantization and before inverse quantization in memory, the amount of data stored in memory can be reduced.

[0147] Furthermore, as a data storage area in the memory, a data storage area corresponding to all nodes may be secured, or a data storage area corresponding to only the existing nodes may be secured. In the latter case, it is possible to reduce the amount of data storage area used.

[0148] Furthermore, the data storage area of ​​the memory may be a data storage area corresponding to all the transform coefficients, or a data storage area corresponding to only the existing transform coefficients. In the latter case, the amount of data storage area used can be reduced.

[0149] 13 is a diagram showing an example of reference frame data stored in memory. In FIG. 13, the layer depth and position information pos of the node are stored along with the conversion coefficient coeff for each node. Note that when an attribute value has multiple components, or when a three-dimensional point has multiple types of attribute values, the respective conversion coefficients are stored. An example of a case where an attribute value has multiple components is when the attribute value is color information and has three components (e.g., an R component, a G component, and a B component). Furthermore, examples of multiple types of attribute values ​​include color information, reflectance, and normal information.

[0150] The transform coefficients shown in FIG. 13 are represented by coeff[j][i]. j ranges from 0 to N-1, and N is determined by the number of components and the number of types of attribute values. For example, if the attribute values ​​are color information, N is 3, which is the number of color information components. If the attribute values ​​are color information and reflectance, N is 4, which is the sum of the number of color information components and the number of reflectance components. The types and components may be represented by separate parameters. i ranges from 0 to 7, which corresponds to the eight frequency components of the transform coefficients.

[0151] Note that a Morton code may be stored instead of the layer depth and the position information pos. In this case, the encoding device 100 can calculate the layer depth and the position information pos using the Morton code.

[0152] The memory may have a single buffer configuration, or a double buffer configuration in which the saving process and the reference process can be performed simultaneously (asynchronously). In the case of a double buffer configuration, the buffer to be used may be switched when switching frames.

[0153] Alternatively, the memory may store the transform coefficients regardless of whether they are valid or invalid, and may further store a flag indicating whether the transform coefficients are valid or invalid. The encoding device 100 determines whether the transform coefficients are valid or invalid by referring to the flag.

[0154] [Syntax Example] An example of syntax and semantics will be described. Fig. 14 is a diagram showing an example of the syntax of APS. As shown in Fig. 14, APS includes inter_raster_prediction_enabled and depth.

[0155] inter_raht_prediction_enabled is a flag indicating whether inter prediction is applied to the attribute value (transform coefficient after RAHT), in other words, the flag indicates whether the inter prediction method is enabled.

[0156] When inter_rf_prediction_enabled indicates that inter prediction is to be applied, APS includes depth. Depth is information indicating the depth to which inter prediction is to be applied. For example, it may be specified that inter prediction is always applied to the highest layer, and depth may indicate the depth -1 to which inter prediction is to be applied. For example, inter prediction is applied to layers above (or equal to or greater than) the depth indicated by depth (or depth -1), and inter prediction is not applied to layers below (or less than) the depth indicated by depth (or depth -1).

[0157] Note that the encoding device 100 may determine whether to apply inter prediction to a frame or slice based on, in addition to inter_raster_prediction_enabled, whether the frame or slice is a random access point, whether inter prediction is applied to position information, or a combination of other conditions. In this case, a flag indicating whether inter prediction of attribute values ​​is applied to the data unit may be stored in the ADUH. That is, information regarding inter prediction of attribute values ​​may be stored in units of sequences, frames, or slices, or in units of multiple of these.

[0158] When switching whether or not to apply inter prediction in units of frames or slices, the above-described inter prediction process is performed for each frame or slice.

[0159] [Modification] Note that, although the above describes an example in which a frame or slice at a different time from the target frame or target slice is referenced, a frame or slice at the same time as the target frame or target slice may also be referenced. For example, when a slice at the same time as the target slice is referenced, a node at the same depth and position as the target node may be referenced. Alternatively, a node that is estimated to have a high correlation, such as a parent node or adjacent node of a node at the same depth and position as the target node, may also be referenced.

[0160] Furthermore, an attribute value that is highly correlated with the target attribute value but different from the target attribute value may be referenced. For example, if there are multiple attribute values ​​with different parallaxes or viewpoints, the attribute value with different parallaxes or viewpoints from the target attribute value may be referenced. Furthermore, if the attribute value has multiple components, other components that are highly correlated with the target component being processed may be referenced. Note that these referenced attribute values ​​or components may be included in a frame or slice at a different time from the target frame or target slice, or may be included in a frame or the same slice at the same time as the target frame or target slice.

[0161] As described above, the encoding device and decoding device according to this embodiment refer to nodes at the same layer and position in inter prediction. This is equivalent to using information at the same spatial position for prediction, and it is expected that the transform coefficients will also have similar properties, improving prediction efficiency.

[0162] Furthermore, if it is possible to search for a node with a matching hierarchy and position, the reference frame does not need to have the same spatial RAHT tree structure, which increases the degree of reference freedom and can be expected to improve prediction efficiency. In other words, even if the child node structures do not match, inter prediction can be performed on the target node, and the effects of inter prediction can be expected.

[0163] Second Embodiment [Overview] In this embodiment, a method for inter-prediction processing of attribute values ​​including motion compensation will be described. In motion compensation, in inter-prediction of transform coefficients after RAHT, the positions of a target frame and a reference frame are corrected based on motion.

[0164] For example, the encoding device corrects position information of at least one of the target point group (e.g., target frame) and the reference point group (e.g., reference frame) so as to increase the correlation between them. For example, if there is movement in the point group between frames at different times, the encoding device corrects position information of one of the target point group and the reference point group based on motion information such as a motion vector. Furthermore, if the target point group and the reference point group have different positions or viewpoint directions, the encoding device may correct position information of one of the target point group and the reference point group based on position difference information such as a spatial vector.

[0165] In this embodiment, the encoding device has a configuration capable of switching between first inter prediction and second inter prediction. The first inter prediction is an inter prediction method without motion compensation, such as the inter prediction method described in the first embodiment.

[0166] The second inter prediction is an inter prediction method with motion compensation. In the second inter prediction, the encoding device corrects position information of the reference points so as to increase the correlation between the target point group and the reference point group. Specifically, the encoding device corrects the position information of the reference points and generates transform coefficients by performing RAHT on attribute values ​​using the position information of the corrected points. The generated transform coefficients are used as predicted values.

[0167] [Configuration of Encoding Device and Decoding Device] Figure 15 is a block diagram showing the configuration of an encoding device 100A according to this embodiment. As shown in Figure 15, the encoding device 100A includes a transform unit 101, a subtraction unit 102, a switching unit 103, a quantization unit 104, an arithmetic coding unit 105, an inverse quantization unit 106, a first inter prediction unit 107, switching units 121 and 122, and a second inter prediction unit 123. Note that elements similar to those of the encoding device 100 shown in Figure 8 are denoted by the same reference numerals, and the following description will mainly focus on differences from the encoding device 100 shown in Figure 8.

[0168] The switching units 121 and 122 are switches for switching between the first inter prediction and the second inter prediction. When the first inter prediction is used, the switching unit 121 outputs the transform coefficients or the difference values ​​output from the inverse quantization unit 106 to the first inter prediction unit 107. When the second inter prediction is used, the switching unit 121 outputs the transform coefficients or the difference values ​​output from the inverse quantization unit 106 to the second inter prediction unit 123.

[0169] When the first inter prediction is used, the switching unit 122 outputs the first predicted value generated by the first inter prediction unit 107 as the third predicted value to the subtraction unit 102. When the second inter prediction is used, the switching unit 122 outputs the second predicted value generated by the second inter prediction unit 123 as the third predicted value to the subtraction unit 102. The subtraction unit 102 calculates a difference value that is the difference between the transform coefficient generated by the transform unit 101 and the third predicted value output from the switching unit 122.

[0170] The second inter prediction unit 123 includes a switching unit 131 , an addition unit 132 , an inverse transformation unit 133 , a frame buffer 134 , a switching unit 135 , a motion compensation unit 136 , a switching unit 137 , and a transformation unit 138 .

[0171] When inter prediction (second inter prediction) is used, the switching unit 131 outputs the difference value obtained by the inverse quantization unit 106 to the addition unit 132, and when inter prediction is not used, the switching unit 131 outputs the transform coefficient obtained by the inverse quantization unit 106 to the inverse transform unit 133. The addition unit 132 generates a transform coefficient by adding the difference value and the second predicted value, and outputs the generated transform coefficient to the inverse transform unit 133.

[0172] The inverse transform unit 133 regenerates attribute values ​​by performing an inverse RAHT (e.g., an inverse Haar transform) on the transform coefficients output from the switching unit 131 or the adding unit 132. Here, the transform coefficients correspond to nodes in the RAHT tree structure, while the regenerated attribute values ​​correspond to position information for each point.

[0173] The frame buffer 134 is a memory that stores attribute values ​​of coded and decoded 3D points. For example, the frame buffer 134 stores attribute values ​​for one frame or one data unit. Note that the frame buffer 134 may store attribute values ​​for multiple frames or multiple data units.

[0174] The switching units 135 and 137 are switches for switching whether or not motion compensation is performed. When motion compensation is performed, the switching unit 135 outputs the attribute values ​​stored in the frame buffer 134 to the motion compensation unit 136, and the switching unit 137 outputs the motion-compensated attribute values ​​generated by the motion compensation unit 136 to the conversion unit 138. When motion compensation is not performed, the attribute values ​​stored in the frame buffer 134 (attribute values ​​that are not motion-compensated) are output to the conversion unit 138.

[0175] The motion compensation unit 136 performs position correction (motion compensation) using a motion vector between frames detected by a motion vector detection unit (not shown). The motion vector is calculated based on, for example, a change in the reference position of the frame. The frame buffer 134 may store attribute values ​​after motion compensation. Alternatively, the second inter prediction unit 123 may further include a frame buffer that stores attribute values ​​after motion compensation. In motion compensation, position information corresponding to the attribute values ​​is corrected. As a result, for example, a node determined to be in the same position as the target node is adjusted.

[0176] The transform unit 138 generates transform coefficients by applying RAHT (e.g., Haar transform) to the attribute values. This RAHT is, for example, similar to the RAHT performed by the transform unit 101. For example, the transform coefficients are output as second predicted values.

[0177] Note that, although an example has been shown in which the encoding device 100A switches between the first inter prediction and the second inter prediction, the encoding device 100A may have a function of performing only the second inter prediction.

[0178] When the first inter prediction is used, since transform coefficients that can be directly used for prediction are stored in memory, the transform processing for prediction is not required, and the inter prediction function can be realized with a simple configuration. In addition, the processing load can be reduced.

[0179] When using the second inter prediction, it is possible to correct the attribute values ​​of the position information. For example, if a point cloud position shift occurs due to motion, spatial inter-frame correction processing can be easily applied using motion compensation. This is expected to improve prediction accuracy even for sequences with large motion.

[0180] Furthermore, when a frame is divided into multiple slices or tiles for processing, the spatial positions of the point clouds between the divided data may differ. Even in such cases, the position information can be corrected, making it easy to combine the divided data. In other words, the entire data before division can be regenerated and used as reference attribute information.

[0181] The memory that stores the attribute values ​​may store attribute values ​​that have been corrected to improve motion compensation or prediction accuracy, or may store attribute values ​​before motion compensation or correction.

[0182] Fig. 16 is a block diagram showing the configuration of a decoding device 200A according to this embodiment. This decoding device 200A generates attribute values ​​of a plurality of 3D points included in point cloud data by decoding, for example, a bit stream generated by the encoding device 100A shown in Fig. 15.

[0183] As shown in Fig. 16, the decoding device 200A includes an arithmetic decoding unit 201, an inverse quantization unit 202, a switching unit 203, an addition unit 204, an inverse transform unit 205, a first inter prediction unit 206, switching units 211, 212, 214, and 215, and a second inter prediction unit 213. Note that the same elements as those in the decoding device 200 shown in Fig. 9 are denoted by the same reference numerals, and the following description will mainly focus on the differences from the decoding device 200 shown in Fig. 9.

[0184] The switching unit 211 is a switch for switching whether to use inter prediction (whether to use inter prediction or intra prediction). When inter prediction is used, the switching unit 211 outputs the transform coefficients generated by the addition unit 204 to the inverse transform unit 205 and the switch 214, and when inter prediction is not used, the switching unit 211 outputs the transform coefficients output from the switch 203 (the transform coefficients obtained by the inverse quantization unit 202) to the inverse transform unit 205 and the switch 214.

[0185] The switching units 212, 214, and 215 are switches for switching between the first inter prediction and the second inter prediction. When the first inter prediction is used, the switching unit 214 outputs the transform coefficients output from the switch 211 (the transform coefficients generated by the adder 204) to the first inter prediction unit 206. When the second inter prediction is used, the switching unit 215 outputs the attribute values ​​generated by the inverse transform unit 205 to the second inter prediction unit 213.

[0186] When the first inter prediction is used, the switching unit 212 outputs the first predicted value generated by the first inter prediction unit 206 as the third predicted value to the addition unit 204. When the second inter prediction is used, the switching unit 212 outputs the second predicted value generated by the second inter prediction unit 213 as the third predicted value to the addition unit 204. The addition unit 204 generates a transform coefficient by adding the difference value generated by the inverse quantization unit 202 and the third predicted value output from the switching unit 212.

[0187] The second inter prediction unit 213 includes a frame buffer 221 , a switching unit 222 , a motion compensation unit 223 , a switching unit 224 , and a conversion unit 225 .

[0188] The frame buffer 221 is a memory that stores attribute values ​​of decoded 3D points. For example, the frame buffer 221 stores attribute values ​​for one frame or one data unit. Note that the frame buffer 221 may store attribute values ​​for multiple frames or multiple data units.

[0189] The switching units 222 and 224 are switches for switching whether or not motion compensation is performed. When motion compensation is performed, the switching unit 222 outputs the attribute values ​​stored in the frame buffer 221 to the motion compensation unit 223, and the switching unit 224 outputs the motion-compensated attribute values ​​generated by the motion compensation unit 223 to the conversion unit 225. When motion compensation is not performed, the attribute values ​​stored in the frame buffer 221 (attribute values ​​that are not motion-compensated) are output to the conversion unit 225.

[0190] The motion compensation unit 223 performs position correction (motion compensation) using a motion vector between frames. For example, information indicating the motion vector is included in the bitstream, and the motion compensation unit 223 performs motion compensation using the motion vector acquired from the bitstream. Note that the frame buffer 221 may store attribute values ​​after motion compensation. Alternatively, the second inter prediction unit 213 may further include a frame buffer that stores attribute values ​​after motion compensation.

[0191] The transform unit 225 generates transform coefficients by applying RAHT (e.g., Haar transform) to the attribute values. This RAHT is, for example, similar to the RAHT performed by the transform unit 138. For example, the transform coefficients are output as second predicted values.

[0192] Note that, although an example has been shown here in which the decoding device 200A switches between the first inter prediction and the second inter prediction, the decoding device 200A may have a function of performing only the second inter prediction.

[0193] Furthermore, the encoding device 100A may store a flag indicating whether the first inter prediction or the second inter prediction will be used in metadata such as SPS, APS, or ADUH. The decoding device 200A may switch between using the first inter prediction and the second inter prediction based on the flag stored in the metadata.

[0194] [Motion Compensation Processing] The processing of motion compensation for three-dimensional points corresponding to attribute values ​​will be described. FIG. 17 is a diagram showing an example of position correction processing for a reference point group. FIG. 17 shows an example in which frame #x, which is a target frame, references frame #x-1. Frame #x-1 is also divided into multiple slices (regions #1 and #2) and encoded. The decoding device decodes the point group for each slice. Frame #x is divided into multiple slices (regions #3 and #4) and encoded. For example, region #3 references region #1, and region #4 references region #2.

[0195] As shown in the left and center diagrams of frame #x-1 in Fig. 17, the original point cloud is generated by shifting the multiple 3D points included in the point cloud for each slice to the positions of the original divided regions for each slice and then combining them. As shown in the right diagram of frame #x-1 in Fig. 17, if motion compensation is required for inter prediction, position correction for motion compensation between frames is also performed.

[0196] The first inter prediction and the second inter prediction may be switched depending on whether the above-described position correction (motion compensation) is necessary. For example, the first inter prediction and the second inter prediction may be switched depending on whether there is motion in the sequence or whether there is a motion vector. For example, the encoding device 100A may use the first inter prediction in scenes with little motion and the second inter prediction in scenes with much motion.

[0197] Alternatively, the encoding device 100A may use the first inter prediction for sequences or content with little motion, and the second inter prediction for sequences or content with much motion. In this manner, the encoding device 100A may switch between the first and second inter predictions for each sequence or content.

[0198] Furthermore, for example, when the effectiveness of motion compensation differs based on the characteristics of a point group, the encoding device 100A may switch between the first inter prediction and the second inter prediction based on the characteristics of the target point group.

[0199] For example, if the target point cloud is a sparse point cloud (a point cloud with a low point density), the encoding device 100A may determine that motion compensation will have little effect and use the first inter prediction. If the target point cloud is a dense point cloud (a point cloud with a high point density), the encoding device 100A may determine that motion compensation will have a large effect and use the second inter prediction. For example, if point clouds from multiple frames are merged in advance to generate a dense point cloud with a large number of points, and the generated dense point cloud is to be encoded, the second inter prediction may be used.

[0200] In addition, when the point cloud is a sparse point cloud and motion compensation is effective, the encoding device 100A may use the second inter prediction for the sparse point cloud. In addition, as other point cloud characteristics, the content type and the point cloud generation method can be used.

[0201] Furthermore, the encoding device 100A may switch between the first inter prediction and the second inter prediction based on whether position correction of the segmentation point group (tile or slice) is necessary. For example, when there is only one slice in a frame, the encoding device 100A may determine that position correction of the segmentation point group is unnecessary and apply the first inter prediction. Furthermore, the encoding device 100A may use the second inter prediction when there are multiple segmentation point groups.

[0202] Alternatively, the encoding device 100A may fix the position or origin of the segmentation point group across multiple frames. Specifically, the region positions of the segmentation point group of the reference frame and the region positions of the segmentation point group of the target frame are specified to match. As a result, the region positions of the segmentation point group of the reference frame and the region positions of the segmentation point group of the target frame always match, eliminating the need for position correction. Therefore, the encoding device 100A may apply first inter prediction in this case. That is, the encoding device 100A may determine whether or not to fix the position or origin of the segmentation point group across multiple frames, and may apply first inter prediction if the position or origin of the segmentation point group is fixed across multiple frames, and may apply second inter prediction if not.

[0203] The encoding device 100A may determine whether to switch between the first inter prediction and the second inter prediction, or whether to perform position correction, for each point cloud data or for each divided data.

[0204] Alternatively, the positions of the nodes may be corrected without inversely transforming the transform coefficients. For example, the encoding device 100A detects a motion vector, calculates a position to which the position of a node included in the RAHT tree will be shifted using the motion vector, and updates the position of the node. Figure 18 is a diagram showing an example of a shift in the position of a node in the RAHT tree. Figure 19 is a diagram showing a three-dimensional area corresponding to the RAHT tree shown in Figure 18.

[0205] For example, as shown in Fig. 18, a certain node is shifted to another position in the RAHT tree. This position shift is calculated based on the motion vector. As a result, the RAHT tree can be updated to correspond to the three-dimensional area after the position correction based on the motion vector, as shown in Fig. 19.

[0206] If the RAHT tree structure of the reference frame cannot represent the node after the position shift, the encoding device 100A may extend the hierarchy of the RAHT tree structure and increase the number of upper hierarchical layers.

[0207] This position shift changes the structure of the RAHT tree from before the position shift. Also, since the data can be buffered as transform coefficients, the above-mentioned effects can be expected.

[0208] [Syntax Example] An example of syntax and semantics will be described. Fig. 20 is a diagram showing an example of the syntax of an APS. As shown in Fig. 20, the APS includes inter_raht_prediction_enabled, depth, raht_inter_prediction_type, and reference_position_compensate.

[0209] inter_raht_prediction_enabled is a flag indicating whether inter prediction is applied to the attribute value (transform coefficient after RAHT), in other words, the flag indicates whether the inter prediction method is enabled.

[0210] When inter_raht_prediction_enabled indicates that inter prediction is to be applied, the APS includes depth, raht_inter_prediction_type, and reference_position_compensate.

[0211] "depth" is information indicating the depth to which inter prediction is applied, and is similar to, for example, "depth" shown in Fig. 14. "raht_inter_prediction_type" is a flag indicating whether first inter prediction or second inter prediction is to be applied. "reference_position_compensate" is a flag indicating whether position correction (motion compensation) of the reference point group is to be performed.

[0212] Note that information relating to inter prediction of these attribute values ​​may be stored in units of sequences, frames, or slices, or may be stored in units of multiple of these.

[0213] Furthermore, in the above description, point cloud data has been cited as an example of three-dimensional data to be processed, but the method of this embodiment can also be applied to other three-dimensional data such as mesh data. Mesh data (also called three-dimensional mesh data) is a data format used in CG (Computer Graphics) and represents the three-dimensional shape of an object as a collection of surface information. Mesh data includes point cloud information (e.g., vertex information). Therefore, for example, when encoding mesh data generated from point cloud data, the method of this embodiment can be applied to encoding this point cloud information.

[0214] [Summary] A decoding device (3D data decoding device) according to an embodiment performs the processing shown in Fig. 21. The decoding device calculates a predicted value of a first coefficient (e.g., a transform coefficient) of a first node included in a first tree structure of a current coding unit (e.g., a current frame or a current slice) by referring to a reference coding unit (e.g., a reference frame or a reference slice) (S201), obtains a residual value (e.g., a difference value or a prediction residual) of the first coefficient from the bitstream (S202), calculates the first coefficient based on the predicted value and the residual value (S203), performs an inverse hierarchical transform (e.g., an inverse RAHT or an inverse Haar transform) on the first coefficient (S204), and calculates attribute values ​​of 3D points included in the current coding unit based on the result of the inverse hierarchical transform (S205).

[0215] According to this, when there is correlation between coefficients between coding units, the accuracy of inter prediction can be improved, and therefore coding efficiency can be improved.

[0216] For example, an inverse hierarchical transform is an inverse transform of a hierarchical transform applied to two coefficients of two adjacent nodes to calculate the coefficients of a higher node located above the two adjacent nodes. In such a hierarchical transform, correlation is likely to occur between coefficients in coding units. Therefore, the accuracy of inter-prediction can be improved, thereby improving coding efficiency.

[0217] In the above-described embodiment, the RAHT and Haar transform are used as examples of hierarchical transforms, but hierarchical transforms are not limited to these. Other transforms may be used as hierarchical transforms as long as the coefficients of a higher layer represent the principal components of the coefficients of a lower layer. For example, hierarchical transforms may be performed in which principal components are calculated from the transform coefficients or attribute values ​​of all nodes belonging to a layer, and the calculated principal components are used as input values ​​for the higher layer.

[0218] For example, in the calculation of the predicted value (S201), a second node included in the second tree structure of the reference coding unit is referenced, and the second position of the second node in the second tree structure is the same as the first position of the first node in the first tree structure. By referencing the second node at the same position as the first node, where there is a high possibility that coefficients between coding units are correlated, the accuracy of inter prediction can be improved. Therefore, the coding efficiency can be improved.

[0219] For example, in the calculation of the predicted value (S201), a second node included in the second tree structure of the reference coding unit is referenced, and the difference between the second position of the second node in the second tree structure and the first position of the first node in the first tree structure is less than a predetermined threshold. This makes it possible to improve prediction accuracy by referencing a second node located close to the first node, which is likely to have a correlation between coefficients between coding units. This, in turn, improves coding efficiency.

[0220] For example, the first and second locations are represented by a Morton code, which can be used to easily search for a second node at the same location as or close to the first node.

[0221] For example, in hierarchical transformation, values ​​of low-frequency components and values ​​of high-frequency components are generated, the low-frequency components correspond to the first coefficients, and the values ​​of the high-frequency components are not predicted. This allows for improved coding efficiency by using prediction for low-frequency components, which tend to have high prediction accuracy. Furthermore, not using prediction for high-frequency components allows for reduced processing load.

[0222] For example, the decoding device further stores the first coefficient in a buffer memory (e.g., the frame buffer 207) for calculating predicted values ​​of coefficients of other nodes, thereby allowing the decoding device to use the first coefficient stored in the buffer memory for predicting coefficients of other nodes.

[0223] For example, the decoding device further obtains a quantized value obtained by quantizing the first coefficient (for example, obtains the quantized value from the bitstream), and stores the quantized value in a buffer memory (for example, the frame buffer 207) for calculating predicted values ​​of other nodes without performing inverse quantization on the quantized value. This makes it possible to reduce the amount of data stored in the memory.

[0224] For example, the reference is made in units of nodes. This allows prediction to be applied when the correlation is partially high even when the correlation is not high overall in the coding unit, thereby improving coding efficiency.

[0225] For example, in the calculation of the predicted value (S201), the second node included in the second tree structure of the reference coding unit is referenced, and if the number of coefficients of the first node is the same as the number of coefficients of the second node, the predicted value is calculated. This makes it possible to prevent erroneous predictions.

[0226] For example, when at least one of the number of 3D points included in the target coding unit, the arrangement of the 3D points, the density of the 3D points, and the depth of the first node in the first tree structure satisfies a predetermined condition, a predicted value is calculated. In this way, when there is a low possibility of correlation between coding units, prediction is not performed, thereby reducing the amount of processing.

[0227] For example, if the condition is satisfied, the first coefficient is stored in a buffer memory (e.g., the frame buffer 207) for calculation of predicted values ​​of coefficients of other nodes, thereby reducing the amount of data stored in the memory.

[0228] For example, in the calculation of the predicted value (S201), motion compensation is performed on the reference coding unit, and the predicted value is calculated by referring to the reference coding unit after motion compensation. This can improve prediction accuracy when there is motion between coding units, thereby improving coding efficiency.

[0229] For example, the decoding device may further store the calculated attribute values ​​in a buffer memory (e.g., the frame buffer 221) for calculating predicted values ​​of coefficients of other nodes, thereby allowing the decoding device to use the attribute values ​​stored in the buffer memory for predicting coefficients of other nodes.

[0230] For example, the decoding device calculates a predicted value of a first coefficient (e.g., a transform coefficient) of a first node included in a target coding unit (e.g., a target frame or a target slice) by referring to a reference coding unit (e.g., a reference frame or a reference slice), obtains a residual value (e.g., a difference value or a prediction residual) of the first coefficient from the bitstream, calculates the first coefficient based on the predicted value and the residual value, and performs an inverse transform process (e.g., an inverse RAHT or an inverse Haar transform) on the first coefficient to calculate an attribute value of a three-dimensional point included in the target coding unit, where the first coefficient is generated by a transform process (e.g., an RAHT or a Haar transform) executed by the encoding device to convert the attribute value into the first coefficient.

[0231] According to this, when there is correlation between coefficients between coding units, prediction accuracy can be improved, and therefore coding efficiency can be improved.

[0232] For example, when calculating a predicted value, the decoding device calculates an attribute value of the reference coding unit by performing an inverse hierarchical transformation on the second coefficient of the reference coding unit, performs motion compensation on the attribute value of the reference coding unit, and calculates the predicted value by performing a hierarchical transformation on the attribute value of the reference coding unit after motion compensation.

[0233] 22 is a block diagram of the decoding device 10. For example, the decoding device 10 includes a processor 11 and a memory 12, and the processor 11 performs the above-described processing using the memory 12.

[0234] Moreover, a coding device (three-dimensional data coding device) according to an embodiment performs the processing shown in Fig. 23. The coding device calculates a first coefficient (e.g., a transform coefficient) of a first node included in a first tree structure of a current coding unit (e.g., a current frame or a current slice) by hierarchical transformation (e.g., RAHT or Haar transformation) of attribute values ​​of three-dimensional points included in the current coding unit (e.g., a current frame or a current slice) (S211), calculates a predicted value of the first coefficient by referring to a reference coding unit (e.g., a reference frame or a reference slice) (S212), calculates a residual value (e.g., a difference value or a prediction residual) that is the difference between the first coefficient and the predicted value (S213), and generates a bitstream including the residual value (S214).

[0235] According to this, when there is correlation between coefficients between coding units, the accuracy of inter prediction can be improved, and therefore coding efficiency can be improved.

[0236] For example, a hierarchical transform is a hierarchical transform applied to two coefficients of two adjacent nodes to calculate the coefficient of a higher node located above the two adjacent nodes. In such a hierarchical transform, correlations tend to appear between the coefficients of the coding units. This improves prediction accuracy and coding efficiency.

[0237] For example, in the calculation of the predicted value (S212), a second node included in the second tree structure of the reference coding unit is referenced, and the second position of the second node in the second tree structure is the same as the first position of the first node in the first tree structure. This makes it possible to improve prediction accuracy by referencing the second node at the same position as the first node, which is likely to have a correlation between coefficients between coding units. This, in turn, improves coding efficiency.

[0238] For example, in the calculation of the predicted value (S212), a second node included in the second tree structure of the reference coding unit is referenced, and the difference between the second position of the second node in the second tree structure and the first position of the first node in the first tree structure is less than a predetermined threshold. This makes it possible to improve prediction accuracy by referencing a second node located close to the first node, which is likely to have a correlation between coefficients between coding units. This, in turn, improves coding efficiency.

[0239] For example, the first and second locations are represented by a Morton code, which can be used to easily search for a second node at the same location as or close to the first node.

[0240] For example, in hierarchical transformation, values ​​of low-frequency components and values ​​of high-frequency components are generated, the low-frequency components correspond to the first coefficients, and the values ​​of the high-frequency components are not predicted. This allows for improved coding efficiency by using prediction for low-frequency components, which tend to have high prediction accuracy. Furthermore, not using prediction for high-frequency components allows for reduced processing load.

[0241] For example, the encoding device further stores the first coefficient in a buffer memory (e.g., the frame buffer 113) for calculating predicted values ​​of coefficients of other nodes, so that the encoding device can use the first coefficient stored in the buffer memory to predict coefficients of other nodes.

[0242] For example, the encoding device further generates a quantized value by quantizing the first coefficient, and stores the quantized value in a buffer memory (e.g., the frame buffer 113) for calculating predicted values ​​of other nodes without dequantizing the quantized value. This makes it possible to reduce the amount of data stored in the memory.

[0243] For example, the reference is made in units of nodes. This allows prediction to be applied when the correlation is partially high even when the correlation is not high overall in the coding unit, thereby improving coding efficiency.

[0244] For example, in the calculation of the predicted value (S212), the second node included in the second tree structure of the reference coding unit is referenced, and if the number of coefficients of the first node is the same as the number of coefficients of the second node, the predicted value is calculated. This makes it possible to prevent erroneous predictions.

[0245] For example, when at least one of the number of 3D points included in the target coding unit, the arrangement of the 3D points, the density of the 3D points, and the depth of the first node in the first tree structure satisfies a predetermined condition, a predicted value is calculated. In this way, when there is a low possibility of correlation between coding units, prediction is not performed, thereby reducing the amount of processing.

[0246] For example, if the condition is satisfied, the first coefficient is stored in a buffer memory (e.g., the frame buffer 113) for calculation of predicted values ​​of coefficients of other nodes, thereby reducing the amount of data stored in the memory.

[0247] For example, in the calculation of the predicted value (S212), motion compensation is performed on the reference coding unit, and the predicted value is calculated by referring to the reference coding unit after motion compensation. This can improve prediction accuracy when there is motion between coding units, thereby improving coding efficiency.

[0248] For example, the encoding device may further store the attribute values ​​in a buffer memory (e.g., frame buffer 134) for use in calculating predicted values ​​of coefficients of other nodes, allowing the encoding device to use the attribute values ​​stored in the buffer memory for predicting coefficients of other nodes.

[0249] 24 is a block diagram of the encoding device 20. For example, the encoding device 20 includes a processor 21 and a memory 22, and the processor 21 uses the memory 22 to perform the above-described processing.

[0250] The above describes the encoding device (three-dimensional data encoding device) and decoding device (three-dimensional data decoding device) according to the embodiments and modifications of the present disclosure, but the present disclosure is not limited to these embodiments.

[0251] Furthermore, each processing unit included in the encoding device, decoding device, etc. according to the above-described embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0252] Furthermore, the integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI may also be used.

[0253] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0254] Furthermore, the present disclosure may be realized as an encoding method (three-dimensional data encoding method) or a decoding method (three-dimensional data decoding method) executed by an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device), etc.

[0255] The present disclosure may also be realized as a program that causes a computer, processor, or device to execute the encoding method or decoding method. The present disclosure may also be realized as a bitstream generated by the encoding method. The present disclosure may also be realized as a recording medium on which the program or the bitstream is recorded. For example, the present disclosure may also be realized as a non-transitory computer-readable recording medium on which the program or the bitstream is recorded.

[0256] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0257] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0258] While the encoding device and decoding device according to one or more aspects have been described based on the embodiments, the present disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the present embodiments and configurations constructed by combining components of different embodiments may also be included within the scope of one or more aspects.

[0259] The present disclosure is applicable to encoding devices and decoding devices.

[0260] 10, 200, 200A Decoding device 11, 21 Processor 12, 22 Memory 20, 100, 100A Encoding device 101, 138, 225 Transform unit 102 Subtraction unit 103, 111, 121, 122, 131, 135, 137, 203, 211, 212, 214, 215, 222, 224 Switching unit 104 Quantization unit 105 Arithmetic coding unit 106, 202 Inverse quantization unit 107, 206 First inter prediction unit 112, 132, 204 Addition unit 113, 134, 207, 221 Frame buffer 123, 213 Second inter prediction unit 133, 205 Inverse transformation unit 136, 223 Motion compensation unit 201 Arithmetic Decoding Unit

Claims

1. calculating a predicted value of a first coefficient of a first node included in a first tree structure of a target coding unit using an inter prediction method that refers to a reference coding unit; obtaining a residual value of the first coefficient from the bitstream; calculating the first coefficient based on the predicted value and the residual value; performing an inverse hierarchical transform on the first coefficients; Calculating attribute values ​​of three-dimensional points included in the target coding unit based on the result of the inverse hierarchical transformation. Decryption method.

2. The inverse hierarchical transform is an inverse transform of a hierarchical transform applied to two coefficients of two adjacent nodes to calculate coefficients of a higher node located above the two adjacent nodes. The decoding method according to claim 1.

3. a second node included in a second tree structure of the reference coding unit is referenced in the calculation of the predicted value; The second position of the second node in the second tree structure is the same as the first position of the first node in the first tree structure. The decoding method according to claim 1.

4. a second node included in a second tree structure of the reference coding unit is referenced in the calculation of the predicted value; A difference between the second position of the second node in the second tree structure and the first position of the first node in the first tree structure is less than a predetermined threshold. The decoding method according to claim 1.

5. The first position and the second position are expressed in Morton code.

5. The decoding method according to claim 3 or 4.

6. The hierarchical transformation generates low-frequency component values ​​and high-frequency component values; the low frequency component corresponds to the first coefficient; The values ​​of the high frequency components are not inter predicted. The decoding method according to claim 2.

7. The decoding method further comprises: The first coefficient is stored in a buffer memory for calculation of predicted values ​​of coefficients of other nodes. The decoding method according to claim 1.

8. The decoding method further comprises: obtaining a quantized value obtained by quantizing the first coefficient; The quantized value is not dequantized, and is stored in a buffer memory for calculation of a predicted value of another node. The decoding method according to claim 1.

9. The reference is made on a node-by-node basis. The decoding method according to claim 1.

10. a second node included in a second tree structure of the reference coding unit is referenced in the calculation of the predicted value; When the number of coefficients of the first node is the same as the number of coefficients of the second node, the predicted value is calculated. The decoding method according to claim 1.

11. The predicted value is calculated when at least one of the number of the plurality of 3D points included in the target coding unit, the arrangement of the plurality of 3D points, the density of the plurality of 3D points, and the depth of the first node in the first tree structure satisfies a predetermined condition. The decoding method according to claim 1.

12. If the condition is met, the first coefficient is stored in a buffer memory for calculation of predicted values ​​of the coefficients of other nodes. The decoding method according to claim 11.

13. In the calculation of the predicted value, motion compensation is performed on the reference coding unit, and the predicted value is calculated by referring to the reference coding unit after the motion compensation. The decoding method according to claim 1.

14. The decoding method further comprises: The calculated attribute values ​​are stored in a buffer memory for calculation of predicted values ​​of coefficients of other nodes. The decoding method according to claim 13.

15. In the calculation of the predicted value, calculating an attribute value of the reference coding unit by performing an inverse hierarchical transform on a second coefficient of the reference coding unit; performing motion compensation on the attribute value of the reference coding unit; The predicted value is calculated by performing hierarchical conversion on the attribute value of the reference coding unit after motion compensation. The decoding method according to claim 1.

16. calculating a predicted value of a first coefficient of a first node included in a current coding unit using an inter prediction method that refers to a reference coding unit; obtaining a residual value of the first coefficient from the bitstream; calculating the first coefficient based on the predicted value and the residual value; Calculating attribute values ​​of 3D points included in the target coding unit by performing an inverse transform process on the first coefficients; The first coefficients are generated by a conversion process executed by the encoding device to convert the attribute values ​​into the first coefficients. Decryption method.

17. Calculating a first coefficient of a first node included in a first tree structure of the target coding unit by hierarchically converting attribute values ​​of the three-dimensional points included in the target coding unit; calculating a predicted value of the first coefficient using an inter prediction method that refers to a reference coding unit; calculating a residual value that is a difference between the first coefficient and the predicted value; Generate a bitstream containing the residual values Encoding method.

18. a processor; a memory; The processor uses the memory to: calculating a predicted value of a first coefficient of a first node included in a first tree structure of a target coding unit using an inter prediction method that refers to a reference coding unit; obtaining a residual value of the first coefficient from the bitstream; calculating the first coefficient based on the predicted value and the residual value; performing an inverse hierarchical transform on the first coefficients; Calculating attribute values ​​of three-dimensional points included in the target coding unit based on the result of the inverse hierarchical transformation. Decryption device.

19. a processor; a memory; The processor uses the memory to: Calculating a first coefficient of a first node included in a first tree structure of the target coding unit by hierarchically converting attribute values ​​of the three-dimensional points included in the target coding unit; calculating a predicted value of the first coefficient using an inter prediction method that refers to a reference coding unit; calculating a residual value that is a difference between the first coefficient and the predicted value; Generate a bitstream containing the residual values Encoding device.