Decoding method, decoding device, and encoding device

Abstract

This decoding method comprises: prioritizing inter prediction over intra prediction in first prediction processing when the depth of a target node storing a three-dimensional point is less than a first threshold value (S501); using a prediction method indicated by control information included in a bit stream in second prediction processing when the depth is less than a second threshold value larger than the first threshold value and equal to or less than the first threshold value (S502); and using intra prediction in third prediction processing when the depth is equal to or less than the second threshold value (S503). For example, in at least one of the first prediction processing, the second prediction processing, and the third prediction processing, none of the intra prediction and the inter prediction may be applied according to a predetermined condition.

Description

Decoding method, decoding device, and encoding device

[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device.

[0002] In the future, devices and services that utilize 3D data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, video distribution, etc. 3D data can be acquired in a variety of ways, including distance sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

[0003] One method of representing three-dimensional data is a representation called a point cloud, which represents the shape of a three-dimensional structure using a group of points in three-dimensional space. In a point cloud, the position and color of the points are stored. Point clouds are expected to become the mainstream method of representing three-dimensional data, but point clouds have a very large amount of data. Therefore, when storing or transmitting three-dimensional data, it is essential to compress the amount of data by encoding, just as with two-dimensional moving images (examples include MPEG-4 AVC or HEVC standardized by MPEG).

[0004] Furthermore, compression of point clouds is partially supported by a public library (Point Cloud Library) that performs processing related to point clouds.

[0005] Furthermore, a technique is known in which three-dimensional map data is used to search for and display facilities located around a vehicle (see, for example, Patent Document 1).

[0006] International Publication No. 2014 / 020663

[0007] In such encoding and decoding methods, it is desirable to be able to improve the encoding efficiency.

[0008] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve encoding efficiency.

[0009] In a decoding method according to one aspect of the present disclosure, in a first prediction process when the depth of a target node storing a three-dimensional point is less than a first threshold, inter prediction is prioritized over intra prediction; in a second prediction process when the depth is less than a second threshold that is greater than the first threshold and is equal to or less than the first threshold, a prediction method indicated by control information included in the bitstream is used; and in a third prediction process when the depth is equal to or less than the second threshold, the intra prediction is used.

[0010] An encoding device according to one aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to prioritize inter-prediction over intra-prediction in a first prediction process when the depth of a target node that stores three-dimensional points is less than a first threshold, to store control information indicating the prediction method used in a bitstream in a second prediction process when the depth is less than a second threshold that is greater than the first threshold and is equal to or less than the first threshold, and to use the intra-prediction in a third prediction process when the depth is equal to or less than the second threshold.

[0011] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve encoding efficiency.

[0012] FIG. 1 is a diagram showing an example of an RAHT target node according to an embodiment. FIG. 2 is a diagram showing a three-dimensional region in an octree structure corresponding to an RAHT target node according to an embodiment. FIG. 3 is a flowchart of a first example of encoding processing according to an embodiment. FIG. 4 is a flowchart of intra prediction processing performed by an encoding device according to an embodiment. FIG. 5 is a flowchart of processing for determining conditions for performing intra prediction according to an embodiment. FIG. 6 is a diagram showing a relationship between layers and applied predictions in the first example according to an embodiment. FIG. 7 is a flowchart of a first example of decoding processing according to an embodiment. FIG. 8 is a flowchart of intra prediction processing performed by a decoding device according to an embodiment. FIG. 9 is a flowchart of a second example of encoding processing according to an embodiment. FIG. 10 is a diagram showing a relationship between layers and applied predictions in the second example according to an embodiment. FIG. 11 is a flowchart of a second example of decoding processing according to an embodiment. FIG. 12 is a flowchart of a third example of encoding processing according to an embodiment. FIG. 13 is a flowchart of inter prediction processing performed by an encoding device according to an embodiment. FIG. 14 is a flowchart of a third example of decoding processing according to an embodiment. FIG. 15 is a flowchart of inter prediction processing by a decoding device according to an embodiment. FIG. 16 is a flowchart of a fourth example of encoding processing according to an embodiment. FIG. 17 is a flowchart of a fourth example of decoding processing according to an embodiment. FIG. 18 is a flowchart of a fifth example of encoding processing according to an embodiment. FIG. 19 is a diagram showing the relationship between layers and applied predictions in the fifth example according to the embodiment. FIG. 20 is a flowchart of a fifth example of decoding processing according to an embodiment. FIG. 21 is a diagram showing an example of syntax for attribute data according to an embodiment. FIG. 22 is a diagram showing an example of syntax for APS according to an embodiment. FIG. 23 is a diagram showing another example of syntax for APS according to an embodiment. FIG. 24 is a diagram for explaining depth_u and depth_l according to an embodiment. FIG. 25 is a diagram showing an example of syntax for raht_inter_node_1 and raht_inter_node_2 according to an embodiment. FIG. 26 is a flowchart of decoding processing according to an embodiment.Fig. 27 is a block diagram of a decoding device according to an embodiment, Fig. 28 is a flowchart of an encoding process according to an embodiment, and Fig. 29 is a block diagram of an encoding device according to an embodiment.

[0013] In a decoding method according to one aspect of the present disclosure, in a first prediction process when the depth of a target node storing a three-dimensional point is less than a first threshold, inter prediction is prioritized over intra prediction; in a second prediction process when the depth is less than a second threshold that is greater than the first threshold and is equal to or less than the first threshold, a prediction method indicated by control information included in the bitstream is used; and in a third prediction process when the depth is equal to or less than the second threshold, the intra prediction is used.

[0014] The inventors have found that three-dimensional data tends to be more suitable for intra prediction as the target node becomes deeper, and more suitable for inter prediction as the target node becomes shallower. A prediction process appropriate for the depth of the target node improves coding efficiency. In this aspect, the prediction process can be switched between three levels of node depth. That is, inter prediction is prioritized for shallow nodes, intra prediction is used (prioritized) for deep nodes, and either inter prediction or intra prediction is used for nodes between these. This may further improve coding efficiency compared to switching prediction processes between two levels of node depth. Therefore, the decoding method of this aspect can appropriately decode a bitstream with improved coding efficiency.

[0015] Depending on the 3D data, prediction may be difficult to achieve. Therefore, for example, in at least one of the first prediction process, the second prediction process, and the third prediction process, neither the intra prediction nor the inter prediction may be applied depending on predetermined conditions. This allows the encoding device to encode some nodes without using a prediction process depending on predetermined conditions, thereby generating a bitstream with improved encoding efficiency. Note that in this disclosure, prediction processing refers to decoding the value of a 3D point to be decoded using the value of a reference 3D point.

[0016] For example, the decoding method may further determine a prediction process to be used for the target node using first information indicating the first threshold and the second threshold included in the bitstream. In this way, the decoding method can determine a prediction process to be used for the target node using the first information.

[0017] For example, the first information may include second information indicating the depth from the shallowest depth of the first threshold and third information indicating the depth from the deepest depth of the second threshold, which can reduce the amount of data in the first information.

[0018] For example, the control information may include a first flag for the target node for which the second prediction process is to be used, and when the first flag indicates a first value, the control information may further include a second flag for the target node, and when the second flag indicates a third value, the intra prediction is used in the prediction process of the target node, when the second flag indicates a fourth value, neither the inter prediction nor the intra prediction is used in the prediction process of the target node, and when the first flag indicates a second value, the second flag may not be included in the bitstream, and the inter prediction is used in the prediction process of the target node. This makes it possible to reduce the amount of data of the control information and the amount of processing in a decoding device when inter prediction is frequently used.

[0019] In addition, a decoding device according to one aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to prioritize inter prediction over intra prediction in a first prediction process when the depth of a target node that stores a three-dimensional point is less than a first threshold, to use a prediction method indicated by control information included in the bitstream in a second prediction process when the depth is less than a second threshold that is greater than the first threshold and is equal to or less than the first threshold, and to use the intra prediction in a third prediction process when the depth is equal to or less than the second threshold.

[0020] According to this, the encoding device generates a bitstream with improved encoding efficiency by using a prediction method suitable for the depth of each node, and the decoding device can appropriately decode the bitstream.

[0021] In addition, an encoding device according to one aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to prioritize inter-prediction over intra-prediction in a first prediction process when the depth of a target node that stores three-dimensional points is less than a first threshold, to store control information indicating the prediction method used in a bitstream in a second prediction process when the depth is less than a second threshold that is greater than the first threshold and is equal to or less than the first threshold, and to use the intra-prediction in a third prediction process when the depth is equal to or less than the second threshold.

[0022] This allows the encoding device to generate a bitstream with improved encoding efficiency by using a prediction method suited to the depth of each node.

[0023] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0024] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0025] (Embodiment) [First Example of Predictive Coding Process of Transform Coefficients] A first example of a method for switching between intra-prediction and inter-prediction for transform coefficients obtained by RAHT by an encoding device (three-dimensional data encoding device) will be described. The encoding device generates a bitstream by, for example, encoding three-dimensional data. The three-dimensional data is, for example, three-dimensional point cloud data (also referred to as point cloud data). A point cloud is a collection of multiple three-dimensional points and indicates the three-dimensional shape of an object. The point cloud data includes position information and attribute information (also referred to as attribute values) of multiple three-dimensional points. The position information indicates the three-dimensional position of each three-dimensional point. Note that the position information may also be referred to as geometry information. For example, the position information is expressed in a Cartesian coordinate system or a polar coordinate system.

[0026] The attribute information indicates, for example, color information, reflectance, transmittance, infrared information, normal vector, time information, etc. One 3D point may have a single piece of attribute information or may have multiple types of attribute information.

[0027] For example, the encoding device encodes the position information using an N-ary tree structure such as an octree. Specifically, in an octree, the target space is divided into eight nodes (subspaces), and 8-bit information (occupancy code) indicating whether or not a point cloud is included in each node is generated. Furthermore, the node including the point cloud is further divided into eight nodes, and 8-bit information indicating whether or not the point cloud is included in each of the eight nodes is generated. This process is repeated until the number of point clouds included in a predetermined layer or node falls below a threshold.

[0028] The encoding device also encodes the attribute information using RAHT (Region Adaptive Hierarchical Transform), which is a type of hierarchical encoding method for attribute information that uses position information of three-dimensional points.

[0029] In RAHT, the encoding device first generates a Morton code based on the position information of the 3D points and sorts the attribute information of the 3D points in Morton code order. Next, the encoding device generates high-frequency components and low-frequency components by applying, for example, a Haar transform to the attribute information of two adjacent 3D points in Morton code order. The obtained frequency components are then used as input values ​​for the next layer (upper layer), and the Haar transform is repeated for each layer to obtain multiple transform coefficients (also referred to as coefficients, coding coefficients, or RAHT transform values).

[0030] Fig. 1 is a diagram showing an example of a RAHT target node, and Fig. 2 is a diagram showing a three-dimensional area in an octree structure corresponding to the RAHT target node shown in Fig. 1.

[0031] In the RAHT method, frequency conversion processing is performed in units of, for example, 2 x 2 x 2 voxels. Figure 2 shows an octree representation of the position information of a point cloud. Processing is performed for each node included in the octree hierarchy shown in Figure 1.

[0032] For example, let us consider a case where a first node, which is a RAHT target node located in octree layer N, is used as an input value. In this case, the lowest frequency component among eight frequency components corresponding to the three-dimensional regions of up to eight child nodes of the first node located in octree layer N+1 is used as an input value, and up to eight transform coefficients (frequency components) corresponding to the three-dimensional region of the first node are output by performing RAHT. The lowest frequency component among the eight output transform coefficients is used as one of the input values ​​in the RAHT of the second node, which is the parent node of the first node. Note that if octree layer N+1 is the lowest layer in the octree layer, the three-dimensional region of the node in that layer is a point, and the value is an attribute value of the point. In other words, in the RAHT transformation of the first node, the attribute values ​​of up to eight child nodes (points) of the first node are used as input values.

[0033] Here, the octree hierarchy is defined as depth, and the hierarchy levels are defined as hierarchy 0, hierarchy 1, hierarchy 2, . . . from the top level.

[0034] 3 is a flowchart of a first example of a process for encoding transform coefficients according to the present embodiment. The process shown in FIG. 3 is repeatedly performed for each node in an octree hierarchy included in a target frame to be encoded. First, the encoding device calculates transform coefficients by performing RAHT on the target node (S101).

[0035] Next, the encoding device determines whether the depth (level) of the octree hierarchy to which the target node belongs is the depth of the inter prediction target (S102). Note that the depth of the inter prediction target may be predetermined or may be adaptively determined according to the characteristics of the point group.

[0036] If the depth to which the target node belongs is the depth to be inter-predicted (Yes in S102), the encoding device determines whether a node at the same position as the target node exists in a reference frame (reference point group) (S103). If the encoding device determines that a node at the same position as the target node exists in the reference frame (Yes in S103), it applies inter-prediction to the target node and performs encoding (S104). For example, the encoding device calculates a predicted value using inter-prediction, calculates a difference value (also called a prediction residual) between the predicted value and a transform coefficient, and generates encoded data (a bitstream) by arithmetic coding (entropy coding) the difference value.

[0037] For example, in inter prediction, the encoding device calculates, as a predicted value, a transform coefficient of a reference node included in a reference frame stored in a memory provided in the encoding device. The memory is also called a reference memory. Here, the reference frame is a frame different from the target frame, for example, a frame at a different time from the target frame. Note that the reference frame may be a different frame at the same time as the target frame. For example, the reference frame may be a frame from a different perspective at the same time as the target frame. Note that, although an example in which a different frame is referenced is shown here, a reference processing unit that is a processing unit different from the target processing unit may also be referenced. Here, the processing unit is a unit into which a frame is divided, for example, a slice or a tile.

[0038] Furthermore, the reference node is, for example, a node among multiple nodes included in the reference frame that is located at the same position as the target node. Note that the reference node is not limited to the node located at the same position as the target node, but may also be a nearby node located close to the target node (within a distance equal to or less than a predetermined value). For example, the reference node may be an adjacent node of the node located at the same position as the target node. In other words, the nearby node may be an adjacent node. Also, the reference node may be multiple nodes. For example, the reference node may be multiple adjacent nodes of the node located at the same position as the target node. In this case, the predicted value may be calculated using the transform coefficients of the multiple reference nodes. Note that whether the position of the target node and the position of the reference node match is determined based on, for example, a Morton code. Whether the target node and the reference node are nearby is determined based on whether the difference between their Morton codes is equal to or less than a predetermined threshold.

[0039] On the other hand, if the encoding device determines that there is no node at the same position as the target node in the reference frame (No in S103), it performs encoding without applying prediction to the target node (S105). That is, the encoding device does not apply either inter prediction or intra prediction to the target node. For example, the encoding device generates encoded data (bitstream) by arithmetic coding (entropy coding) of the transform coefficients.

[0040] Furthermore, if the depth to which the target node belongs is not the depth to be subjected to inter prediction (No in S102), the encoding device performs intra prediction processing (S106).

[0041] 4 is a flowchart of the intra prediction process (S106). First, the encoding device determines whether to perform intra prediction by determining whether the conditions for performing intra prediction are satisfied (S111). If the conditions for performing intra prediction are satisfied (Yes in S111), the encoding device performs encoding using intra prediction (S112).

[0042] Here, intra prediction is a prediction process that uses information on other nodes included in a target frame that includes a target node. For example, in intra prediction, an encoding device calculates a predicted value from attribute information on nodes neighboring the target node. Next, the encoding device calculates predicted transform coefficients by performing RAHT on the predicted value. Next, the encoding device calculates a difference value (prediction residual) that is the difference between the transform coefficient obtained by performing RAHT on the target node and the predicted transform coefficient. Next, the encoding device generates coded data (bitstream) by arithmetic coding (entropy coding) the difference value.

[0043] On the other hand, if the conditions for performing intra prediction are not satisfied (No in S111), the encoding device performs encoding without applying prediction to the target node (S113). For example, the encoding device generates encoded data (bit stream) by arithmetic coding (entropy coding) of the transform coefficients.

[0044] In step S111, the encoding device determines whether to perform intra prediction using, for example, node density. Specifically, the encoding device determines to perform intra prediction when the density is high, and determines not to perform intra prediction when the density is low. For example, the encoding device makes this determination using the number of neighboring nodes of the grandparent node and parent node. Figure 5 is a flowchart of this determination process (S111).

[0045] First, the encoding device determines whether the number of neighboring nodes of the grandparent node of the target node is equal to or greater than a first threshold (S121). Note that a neighboring node is a node that includes one or more points located near the target node (the grandparent node in the above example) (e.g., at a distance less than a predetermined value from the target node).

[0046] If the number of neighboring nodes of the grandparent node is equal to or greater than a first threshold (Yes in S121), the encoding device determines whether the number of neighboring nodes of the parent node is equal to or greater than a second threshold (S122).If the number of neighboring nodes of the parent node is equal to or greater than the second threshold (Yes in S122), the encoding device determines that the condition for performing intra prediction is satisfied (S123).

[0047] On the other hand, if the number of neighboring nodes of the grandparent node is less than the first threshold (No in S121), or if the number of neighboring nodes of the parent node is less than the second threshold (No in S122), the encoding device determines that the condition for performing intra prediction is not satisfied (S124). Note that one of steps S121 and S122 may be omitted. Therefore, regardless of whether intra prediction is performed, the number of neighboring nodes of the grandparent node or parent node is compared with the threshold.

[0048] 3, after step S104, S105, or S106, the encoding device stores the transform coefficients of the target node in memory for inter prediction (S107). Note that if the depth (layer) to which the target node belongs is not a depth targeted for inter prediction (a depth to which intra prediction processing can be applied), the encoding device does not need to store the transform coefficients of the target node in memory for inter prediction. This allows the amount of data stored in memory to be reduced.

[0049] In this way, the encoding device determines that inter prediction is applicable when the depth of the target node is the depth of the inter prediction target, and determines that intra prediction is applicable when the depth of the target node is not the depth of the inter prediction target. In other words, the encoding device always prioritizes inter prediction when the depth of the target node satisfies a predetermined condition.

[0050] 6 is a diagram showing the relationship between layers (depths) and applied predictions in Example 1. As shown in Fig. 6, for example, inter prediction may be applied in layers higher than layer α, and intra prediction may be applied in layers lower than layer α.

[0051] Here, a hierarchical coding method using RAHT has the characteristic that transform coefficients in higher layers with shallower depths indicate lower-frequency components of attribute information of 3D points included in the target frame, while transform coefficients in deeper layers indicate higher-frequency components. Generally, low-frequency components of attribute information such as color or reflectance change little over time, so prediction accuracy can be improved by using inter-prediction on the low-frequency components. On the other hand, high-frequency components of attribute information are susceptible to noise and other factors and change significantly over time, making it difficult to improve prediction accuracy using inter-prediction. Therefore, the coding device applies inter-prediction to higher layers containing many low-frequency components of attribute information, or does not perform prediction at all. Then, the coding device does not apply inter-prediction to lower layers containing many high-frequency components, but instead applies intra-prediction, which predicts from neighboring nodes within a frame, or does not perform prediction at all. This increases the likelihood of improving coding efficiency.

[0052] Furthermore, since intra prediction is also effective for predicting high-frequency components if nearby nodes are dense, applying intra prediction to lower layers with many high-frequency components can improve coding efficiency. Note that, as shown in Figure 5, the coding device does not need to apply intra prediction if nearby nodes are not dense, even in lower layers. This makes it possible to balance reduction in processing volume and coding efficiency.

[0053] Furthermore, the number of neighboring nodes of the grandparent node or parent node included in the conditions for performing intra prediction used in the process shown in Fig. 5 is unnecessary when the depth of the target node is the depth of the target for inter prediction, and therefore does not need to be calculated. This reduces the amount of processing required to calculate the number of neighboring nodes.

[0054] Furthermore, the encoding device may prohibit inter prediction for a predetermined target frame and apply intra prediction, or may not perform prediction at all. This allows the encoding device to encode and decode the target frame without relying on information between frames, and the predetermined target frame can be used as a random access point. Note that if the target frame is a random access point, the encoding device may not need to add information required for inter prediction, such as information indicating the depth of the inter prediction target, to the bitstream. This reduces the amount of data in the bitstream.

[0055] [First Example of Predictive Decoding Process of Transform Coefficients] A first example of a method for switching a prediction method (intra prediction or inter prediction) by a decoding device (3D data decoding device) corresponding to the encoding device will be described. The decoding device, for example, decodes a bitstream generated by the encoding device. For example, the decoding device decodes position information using an N-ary tree structure such as an octree. The decoding device also decodes attribute information using an inverse RAHT, which is an inverse transform of the RAHT. The inverse RAHT is a type of hierarchical decoding method for attribute information using position information of three-dimensional points. The inverse RAHT converts transform coefficients into attribute information.

[0056] 7 is a flowchart of a first example of a process for decoding transform coefficients according to this embodiment. The process shown in FIG. 7 is repeatedly performed for each node in an octree hierarchy included in a target frame to be decoded. Note that the process for determining the type of prediction (inter prediction, intra prediction, or no prediction) to be applied to the target node is the same as that performed by the encoding device described above.

[0057] First, the decoding device determines whether the depth of the octree hierarchy to which the target node belongs is the depth of the target of inter prediction (S201).

[0058] If the depth to which the target node belongs is the depth to be subjected to inter prediction (Yes in S201), the decoding device determines whether a node at the same position as the target node exists in the reference frame (S202). If the decoding device determines that a node at the same position as the target node exists in the reference frame (Yes in S202), the decoding device applies inter prediction to the target node and performs decoding (S203). For example, the decoding device generates a difference value of the target node by performing arithmetic decoding (entropy decoding) on ​​the encoded data of the target node included in the bitstream. Next, the decoding device calculates a predicted value using inter prediction and generates a transform coefficient by adding the predicted value and the difference value. The method of calculating the predicted value for inter prediction is the same as the process performed by the encoding device.

[0059] On the other hand, if the decoding device determines that there is no node at the same position as the target node in the reference frame (No in S202), it performs decoding without applying prediction to the target node (S204). That is, the decoding device does not apply either inter prediction or intra prediction to the target node. For example, the decoding device generates transform coefficients of the target node by arithmetic decoding (entropy decoding) of the encoded data of the target node included in the bitstream.

[0060] Furthermore, if the depth to which the target node belongs is not the depth to be subjected to inter prediction (No in S201), the decoding device performs intra prediction processing (S205).

[0061] FIG. 8 is a flowchart of the intra prediction process (S205). First, the decoding device determines whether to perform intra prediction by determining whether the conditions for performing intra prediction are satisfied (S211). If the conditions for performing intra prediction are satisfied (Yes in S211), the decoding device performs decoding using intra prediction (S212). For example, the decoding device generates a difference value of the target node by performing arithmetic decoding (entropy decoding) on ​​the encoded data of the target node included in the bitstream. Next, the decoding device calculates a predicted value using intra prediction and calculates a predicted transform coefficient by performing RAHT on the predicted value. The decoding device generates a transform coefficient by adding the predicted transform coefficient and the difference value. The method for calculating the predicted value of intra prediction is the same as the process in the encoding device.

[0062] On the other hand, if the condition for performing intra prediction is not satisfied (No in S211), the decoding device performs decoding without applying prediction to the target node (S213). For example, the decoding device generates transform coefficients of the target node by performing arithmetic decoding (entropy decoding) on ​​the encoded data of the target node included in the bitstream. Note that, in step S211, for example, a method similar to that shown in FIG. 5 may be used. Alternatively, in step S211, the determination may be made according to control information included in the bitstream.

[0063] 7, after step S203, S204, or S205, the decoding device stores the transform coefficients of the target node in memory for inter prediction (S206). Note that if the depth (layer) to which the target node belongs is not the depth to be subjected to inter prediction (if the depth is to which intra prediction processing is applied), the decoding device does not need to store the transform coefficients of the target node in memory for inter prediction. This allows the amount of data stored in memory to be reduced.

[0064] Finally, the decoding device generates attribute information of the target node by performing inverse RAHT on the transform coefficients obtained in step S203, S204, or S205 (S207).

[0065] As a result, the decoding device can properly decode the bitstream generated by the encoding device.

[0066] Note that, if information indicating that the target frame is a random access point is included in the bitstream, the decoding device may apply intra prediction to decode the target frame. This allows the decoding device to decode the target frame and generate a random access point without relying on inter-frame information. Note that, if the target frame is a random access point, the decoding device does not need to decode information required for inter prediction, such as information indicating the depth of the inter prediction target, from the bitstream. This reduces the amount of processing.

[0067] [Second Example of Predictive Coding Process of Transform Coefficients] A second example of a method for switching prediction methods by an encoding device will now be described. FIG. 9 is a flowchart of the second example of the encoding process of transform coefficients. The process shown in FIG. 9 is performed repeatedly, for example, for each node of an octree hierarchy included in a target frame to be encoded. Note that the following mainly describes differences from the first example shown in FIG. 3. In the process shown in FIG. 9, step S103 is changed to step S103A and step S105 is deleted from the process shown in FIG. 3.

[0068] In the process shown in Figure 9, if the depth to which the target node belongs is the depth to be inter-predicted (Yes in S102) and it is determined that there is no node at the same position as the target node in the reference frame (No in S103A), the encoding device performs intra prediction processing (S106). As a result, even if inter prediction cannot be applied to a shallow upper layer, the encoding device can apply intra prediction, which makes predictions from neighboring nodes within a frame, which may improve encoding efficiency. Since intra prediction is also effective for predicting low-frequency components if neighboring nodes are dense, applying intra prediction instead of inter prediction when inter prediction cannot be applied may improve encoding efficiency.

[0069] 10 is a diagram showing the relationship between layers (depths) and applied predictions in the second example. Note that intra prediction includes no prediction. As shown in FIG. 10 , for example, inter prediction or intra prediction may be applied in layers higher than layer α. Intra prediction may be applied in layers lower than layer α.

[0070] 6 and 10, the prediction method is switched depending on whether the layer is higher than layer α, but the prediction method may be switched for each layer. However, as described above, the prediction method includes no prediction.

[0071] Note that the determination in step S102 may not be performed. Alternatively, all layers may be set as the depth of inter-prediction targets. In this case, the encoding device determines whether inter-prediction is applicable to all layers (e.g., whether a node at the same position as the target node exists in the reference frame). If inter-prediction is applicable, inter-prediction is applied; otherwise, intra-prediction is performed. For example, the encoding device can include all layers in the range of inter-prediction targets by setting the value of depth information added to the bitstream, which indicates the depth of inter-prediction targets, to a value greater than the maximum number of layers in hierarchical encoding using RAHT. This allows the encoding device to improve coding efficiency by applying inter-prediction to all layers for frames included in scenes with high inter-prediction accuracy, such as still scenes. Note that any method may be used to determine whether a scene is still. For example, the encoding device may determine that a scene is still if the value of the overall motion vector between frames is smaller than a predetermined threshold.

[0072] [Second Example of Predictive Decoding Process of Transform Coefficients] A second example of a method for switching prediction methods by a decoding device corresponding to the encoding device according to the second example above will be described. FIG. 11 is a flowchart of the second example of the decoding process of transform coefficients. The process shown in FIG. 11 is performed repeatedly, for example, for each node of an octree hierarchy included in a target frame to be decoded. Note that the following mainly describes differences from the first example shown in FIG. 7. In the process shown in FIG. 11, step S202 is changed to step S202A and step S204 is deleted from the process shown in FIG. 7.

[0073] 11 , if the depth to which the target node belongs is the depth to be subjected to inter prediction (Yes in S201) and it is determined that no node at the same position as the target node exists in the reference frame (No in S202A), the decoding device performs intra prediction processing (S205). As a result, even if inter prediction cannot be applied to an upper layer with a shallow depth, the decoding device can appropriately decode a bitstream with improved coding efficiency by applying intra prediction, which makes predictions from neighboring nodes within the frame.

[0074] Note that the determination in step S201 does not have to be performed. Alternatively, when decoding information in which all layers are set as the depth to be subjected to inter prediction, the decoding device may determine whether inter prediction is applicable to all layers (e.g., whether a node at the same position as the target node exists in the reference frame), and if inter prediction is applicable, apply inter prediction; otherwise, perform intra prediction. For example, if the value of depth information indicating the depth to be subjected to inter prediction decoded from the bitstream is greater than the maximum number of layers in hierarchical encoding by RAHT, the decoding device may determine that all layers are within the range to be subjected to inter prediction. This allows the decoding device to properly decode the bitstream generated by the encoding device.

[0075] [Third Example of Predictive Coding Process of Transform Coefficients] A third example of a method for switching prediction methods by an encoding device will now be described. The third example differs from the first example shown in Figures 3 and 4 in that intra prediction is performed with priority over inter prediction.

[0076] 12 is a flowchart of a third example of a process for encoding transform coefficients. The process shown in FIG. 12 is repeatedly performed for each node in an octree hierarchy included in a target frame to be encoded. First, the encoding device calculates transform coefficients by performing RAHT on the target node (S131).

[0077] Next, the encoding device determines whether to perform intra prediction by determining whether the conditions for performing intra prediction are satisfied (S132). For example, this determination can be made using the same method as in step S111 shown in FIG.

[0078] If the conditions for performing intra prediction are satisfied (Yes in S132), the encoding device performs encoding using intra prediction (S133). On the other hand, if the conditions for performing intra prediction are not satisfied (No in S132), the encoding device performs inter prediction processing (S134).

[0079] 13 is a flowchart of the inter prediction process (S134). First, the encoding device determines whether a node at the same position as the target node exists in the reference frame (reference point group) (S141). If the encoding device determines that a node at the same position as the target node exists in the reference frame (Yes in S141), it applies inter prediction to the target node and performs encoding (S142).

[0080] On the other hand, if the encoding device determines that there is no node in the reference frame at the same position as the target node (No in S141), it performs encoding without applying prediction to the target node (S143).

[0081] Next, as shown in FIG. 12, after step S133 or S134, the encoding device stores the transform coefficients of the current node in a memory for inter prediction (S135).

[0082] In this way, after performing RAHT, the encoding device performs intra prediction if the conditions for performing intra prediction are met, and performs inter prediction if they are not. This makes it possible to improve encoding efficiency by performing intra prediction if possible in a scene with fast motion between frames where inter prediction is difficult, and generating predicted values ​​using inter prediction if intra prediction is not possible.

[0083] Note that the encoding device may switch between a process that prioritizes inter prediction as in the first example and a process that prioritizes intra prediction as in the third example, on a sequence-by-sequence, frame-by-frame, or slice-by-slice basis. For example, the encoding device adds raht_inter_priority_enable, which is information indicating whether inter prediction is prioritized (whether inter prediction or intra prediction is prioritized), to the bitstream. When the value of raht_inter_priority_enable is 1, the encoding device prioritizes inter prediction as in the first example, and when the value of raht_inter_priority_enable is 0, the encoding device prioritizes intra prediction as in the third example.

[0084] For example, in the case of a scene with fast motion, the encoding device sets raht_inter_priority_enable to the value 0 to prioritize intra prediction, and otherwise sets raht_inter_priority_enable to the value 1 to prioritize inter prediction. This allows the encoding device to switch the prioritized prediction method to a prediction method suitable for the scene, thereby improving encoding efficiency.

[0085] Note that in the process shown in Fig. 13, step S141 may include a determination of whether the depth of the target node is the depth targeted for inter prediction, similar to step S102 in Fig. 3. In this case, if the depth of the target node is the depth targeted for inter prediction and a node at the same position as the target node exists in the reference frame, the encoding device applies inter prediction to the target node and performs encoding (S142); otherwise, the encoding device performs encoding without applying prediction to the target node (S143). This allows the encoding device to improve encoding efficiency by applying inter prediction to higher layers that include many low-frequency components that change little over time.

[0086] [Third Example of Predictive Decoding Process of Transform Coefficients] A third example of a method for switching prediction methods by a decoding device corresponding to the encoding device according to the third example above will be described. Fig. 14 is a flowchart of the third example of the decoding process of transform coefficients. The process shown in Fig. 14 is performed repeatedly for each node of an octree hierarchy (eight-ary tree hierarchy) included in a target frame to be decoded, for example.

[0087] First, the decoding device determines whether to perform intra prediction by determining whether a condition for performing intra prediction is satisfied (S231). For example, this determination can be made using the same method as in step S211 shown in FIG. 8.

[0088] If the conditions for performing intra prediction are satisfied (Yes in S231), the decoding device performs decoding using intra prediction (S232). On the other hand, if the conditions for performing intra prediction are not satisfied (No in S231), the decoding device performs inter prediction processing (S233).

[0089] 15 is a flowchart of the inter prediction process (S233). First, the decoding device determines whether a node at the same position as the target node exists in the reference frame (reference point group) (S241). If the decoding device determines that a node at the same position as the target node exists in the reference frame (Yes in S241), it applies inter prediction to the target node and performs decoding (S242).

[0090] On the other hand, if the decoding device determines that there is no node at the same position as the target node in the reference frame (No in S241), it performs decoding without applying prediction to the target node (S243).

[0091] 14, after step S232 or S233, the decoding device stores the transform coefficients of the target node in a memory for inter prediction (S234). Finally, the decoding device generates attribute information of the target node by performing inverse RAHT on the transform coefficients obtained in step S232 or S233 (S235).

[0092] In this way, in the third example, the decoding device prioritizes intra prediction over inter prediction. That is, the decoding device performs intra prediction when the conditions for performing intra prediction are met, and performs inter prediction when the conditions for performing intra prediction are not met. This allows the decoding device to properly decode the bitstream generated by the encoding device.

[0093] Note that the decoding device may switch between a process that prioritizes inter prediction as in the first example and a process that prioritizes intra prediction as in the third example, on a sequence-by-sequence, frame-by-frame, or slice-by-slice basis. For example, the decoding device decodes raht_inter_priority_enable, which is information indicating whether inter prediction is prioritized (whether inter prediction or intra prediction is prioritized), from the bitstream. When the value of raht_inter_priority_enable is 1, the decoding device may prioritize inter prediction as in the first example, and when the value of raht_inter_priority_enable is 0, the decoding device may prioritize intra prediction as in the third example. This allows the decoding device to appropriately decode the bitstream generated by the encoding device.

[0094] Note that in the process shown in Fig. 15, step S241 may include determining whether the depth of the target node is the depth to be subjected to inter prediction, similar to step S201 in Fig. 7. In this case, if the depth of the target node is the depth to be subjected to inter prediction and a node at the same position as the target node exists in the reference frame, the decoding device applies inter prediction to the target node and performs decoding (S242); otherwise, the decoding device performs decoding without applying prediction to the target node (S243). This allows the decoding device to properly decode a bitstream with improved coding efficiency by applying inter prediction to a higher layer that includes many low-frequency components that change little over time.

[0095] [Fourth Example of Predictive Coding Process of Transform Coefficients] A fourth example of a method for switching prediction methods by an encoding device will be described below. Fig. 16 is a flowchart of the fourth example of the encoding process of transform coefficients. The process shown in Fig. 16 is performed, for example, repeatedly for each node in an octree hierarchy included in a target frame to be encoded. First, the encoding device calculates transform coefficients by performing RAHT on the target node (S151).

[0096] Next, the encoding device performs intra prediction processing (S152). For example, the intra prediction processing (S152) is similar to the processing shown in Fig. 4. Next, the encoding device performs inter prediction processing (S153). For example, the inter prediction processing (S153) is similar to the processing shown in Fig. 13.

[0097] Next, the encoding device compares the code amount of the encoded data when intra prediction obtained in step S152 is applied with the code amount of the encoded data when inter prediction obtained in step S153 is applied (S154).

[0098] If the code amount of the coded data when inter prediction is applied is smaller than the code amount of the coded data when intra prediction is applied (Yes in S154), the coding device sets raht_inter_node to the value 1 and stores raht_inter_node in the bitstream (S155). Next, the coding device stores the difference value obtained by inter prediction in step S153 in the bitstream (S156).

[0099] On the other hand, if the code amount of the coded data when intra prediction is applied is smaller than the code amount of the coded data when inter prediction is applied (No in S154), the coding device sets raht_inter_node to the value 0 and stores raht_inter_node in the bitstream (S157). Next, the coding device stores the difference value obtained by intra prediction in step S152 in the bitstream (S158).

[0100] Next, the encoding device stores the transform coefficients of the current node in memory for inter prediction (S159).

[0101] In this way, in the fourth example, the encoding device determines, for example, whether to apply intra prediction or inter prediction to the transform coefficients for each node, and adds information (raht_inter_node) indicating which prediction has been applied to the bitstream. For example, when the value of raht_inter_node is 1, this indicates that inter prediction has been applied to the target node, and when the value of raht_inter_node is 0, this indicates that intra prediction has been applied to the target node.

[0102] This allows the decoding device to determine whether intra prediction or inter prediction should be applied to the target node by decoding raht_inter_node, and to appropriately decode the bitstream.

[0103] In step S154, the encoding device may, for example, compare the code amount of the encoded data obtained by arithmetic coding of the difference value between inter prediction and intra prediction, and select a prediction method with a smaller code amount. This allows for an appropriate selection of a prediction method with a smaller code amount for each node, thereby improving encoding efficiency. Alternatively, the encoding device may compare the code amount of the difference value before arithmetic coding between inter prediction and intra prediction, and select a prediction method with a smaller code amount. This reduces the amount of processing required to calculate the code amount. Furthermore, the code amount used for the determination may include the code amount of raht_inter_node in addition to the code amount of the difference value. This allows for a selection of a prediction method that balances the code amount of the difference value and the code amount of raht_inter_node, thereby improving encoding efficiency.

[0104] Note that, although an example in which raht_inter_node is added to the bitstream for each node has been shown here, this is not necessarily limited to this. For example, for each transform coefficient, information indicating a prediction method (intra prediction or inter prediction) to be applied to the transform coefficient may be added to the bitstream. This makes it possible to improve coding efficiency by switching an appropriate prediction method for each transform coefficient. Furthermore, for example, for each layer, information indicating a prediction method (intra prediction or inter prediction) to be applied to the layer may be added to the bitstream. This makes it possible to improve coding efficiency by switching an appropriate prediction method for each layer.

[0105] [Fourth Example of Predictive Decoding Process of Transform Coefficients] A fourth example of a method for switching prediction methods by a decoding device corresponding to the encoding device according to the fourth example above will be described. Fig. 17 is a flowchart of the fourth example of the decoding process of transform coefficients. The process shown in Fig. 17 is performed repeatedly for each node of an octree hierarchy (eight-ary tree hierarchy) included in a target frame to be decoded, for example.

[0106] First, the decoding device decodes (obtains) information (rht_inter_node) indicating whether the transform coefficients for each node have been coded using intra prediction or inter prediction from the bitstream (S251).

[0107] Next, the decoding device refers to raht_inter_node and determines whether the target node should be decoded using intra prediction or inter prediction (S252). Specifically, if raht_inter_node=1 (Yes in S252), the decoding device determines that the target node has been coded using inter prediction, and performs inter prediction processing (S253). For example, the inter prediction processing (S253) is the same as the processing shown in FIG. 15.

[0108] On the other hand, if raht_inter_node=0 (No in S252), the decoding device determines that the target node has been coded using intra prediction, and performs intra prediction processing (S254). For example, the intra prediction processing (S254) is the same as the processing shown in FIG. 8 .

[0109] Next, the decoding device stores the transform coefficients of the target node in a memory for inter prediction (S255). Finally, the decoding device generates attribute information of the target node by performing inverse RAHT on the transform coefficients obtained in step S253 or S254 (S256).

[0110] In this way, the decoding device can appropriately decode a bitstream with improved coding efficiency by selecting an appropriate prediction method for each node.

[0111] Note that, although an example of decoding a bitstream to which raht_inter_node is added for each node has been shown here, this is not necessarily limited to this. For example, the decoding device may decode, for each transform coefficient, information indicating a prediction method (intra prediction or inter prediction) to be applied to the transform coefficient from the bitstream. This allows the decoding device to properly decode a bitstream in which coding efficiency has been improved by switching an appropriate prediction method for each transform coefficient. Furthermore, for example, the decoding device may decode, for each layer, information indicating a prediction method (intra prediction or inter prediction) to be applied to the layer from the bitstream. This allows the decoding device to properly decode a bitstream in which coding efficiency has been improved by switching an appropriate prediction method for each layer.

[0112] [Fifth Example of Predictive Coding Process of Transform Coefficients] A fifth example of a method for switching prediction methods by an encoding device will be described below. The fifth example is a method that combines the second example of the prediction method switching method (also referred to as inter-priority prediction processing) and the fourth example of the prediction method switching method (also referred to as switching prediction processing).

[0113] 18 is a flowchart of a second example of the process of encoding transform coefficients. The process shown in FIG. 18 is performed repeatedly for each node of an octree hierarchy included in a target frame to be encoded, for example.

[0114] First, the encoding device calculates transform coefficients by performing RAHT on the target node (S301).

[0115] Next, the encoding device determines whether the depth (level) of the octree hierarchy to which the target node belongs is the depth of the inter-priority prediction target (S302). Note that the depth of the inter-priority prediction target may be predetermined or may be adaptively determined according to the characteristics of the point group. Furthermore, information indicating the determined depth of the inter-priority prediction target may be stored in the bitstream.

[0116] If the depth to which the target node belongs is the depth to which inter-priority prediction is to be performed (Yes in S302), the encoding device applies inter-priority prediction processing to the target node (S303).

[0117] Here, the inter-priority prediction process is the method described in the second example of the prediction method switching method above, which is a method of applying inter prediction with priority over intra prediction. Specifically, the encoding device determines whether a node at the same position as the target node exists in the reference frame (reference point group) (S311). If the encoding device determines that a node at the same position as the target node exists in the reference frame (Yes in S311), it applies inter prediction to the target node and performs encoding (S312).

[0118] On the other hand, if the encoding device determines that there is no node at the same position as the target node in the reference frame (No in S311), the encoding device performs intra prediction processing (S313).

[0119] The details of the processes in steps S311 to S313 are the same as, for example, steps S103A, S104, and S106 shown in FIG.

[0120] On the other hand, if the depth to which the target node belongs is not the depth of the inter-priority prediction target (No in S302), the encoding device determines whether to apply switching prediction processing. Specifically, the encoding device determines whether the depth of the octree hierarchy to which the target node belongs is the depth of the switching prediction target (S304). Note that the depth of the switching prediction target may be predetermined or may be adaptively determined according to the characteristics of the point cloud. Furthermore, information indicating the determined depth of the switching prediction target may be stored in the bitstream.

[0121] If the depth to which the target node belongs is the depth to be subjected to switching prediction (Yes in S304), the encoding device applies switching prediction processing to the target node (S305). Here, the switching prediction processing is the method described in the fourth example of the prediction method switching method described above, which determines whether to encode the transform coefficients for each node using intra prediction or inter prediction, and stores information (raht_inter_node) indicating which method was used in the bitstream. For example, a value of raht_inter_node of 1 indicates that the target node has been encoded using inter prediction, and a value of raht_inter_node of 0 indicates that the target node has been encoded using intra prediction.

[0122] Specifically, the encoding device performs intra-prediction processing (S321). Next, the encoding device performs inter-prediction processing (S322). Next, the encoding device compares the code amount of the encoded data obtained in step S321 when intra-prediction is applied with the code amount of the encoded data obtained in step S322 when inter-prediction is applied (S323).

[0123] If the code amount of the coded data when inter prediction is applied is smaller than the code amount of the coded data when intra prediction is applied (Yes in S323), the coding device sets raht_inter_node to the value 1 and stores raht_inter_node in the bitstream (S324). Next, the coding device stores the difference value obtained by inter prediction in step S322 in the bitstream (S325).

[0124] On the other hand, if the code amount of the coded data when intra prediction is applied is smaller than the code amount of the coded data when inter prediction is applied (No in S323), the coding device sets raht_inter_node to the value 0 and stores raht_inter_node in the bitstream (S326). Next, the coding device stores the difference value obtained by intra prediction in step S321 in the bitstream (S327).

[0125] The details of the processes in steps S321 to S327 are the same as, for example, steps S152 to S158 shown in FIG.

[0126] Note that in step S305, the determination of step S323 may be performed only when both intra prediction and inter prediction are applicable. For example, intra prediction is applicable when the conditions for performing intra prediction are satisfied using the methods shown in FIGS. 4 and 5. Inter prediction is applicable when, for example, a node at the same position as the target node exists in the reference frame. Specifically, when only intra prediction is applied, step S323 may not be performed, and steps S326 and S327 may be performed. When only inter prediction is applied, step S323 may not be performed, and steps S324 and S325 may be performed. Furthermore, in these cases, steps S324 and S325 may not be performed, and raht_inter_node may not be added to the bitstream. In this case, the decoding device may perform a similar determination to determine whether to use intra prediction or inter prediction.

[0127] If the depth to which the target node belongs is not the depth to be switched (No in S304), the encoding device applies intra prediction processing to the target node (S306). Note that the details of step S306 are the same as, for example, step S313.

[0128] After step S303, S305, or S306, the encoding device stores the transform coefficients of the current node in memory for inter prediction (S307).

[0129] In this way, the encoding device can select the optimal prediction method according to the characteristics of the target node by applying an appropriate prediction method or combination of prediction methods depending on the depth of the node, thereby improving encoding efficiency.

[0130] Fig. 19 is a diagram showing the relationship between layers (depths) and applied predictions in Example 5. As shown in Fig. 19, for example, inter-priority prediction is applied to layers higher than layer α. Switching prediction is applied to layers lower than layer α and higher than layer β. Intra prediction is applied to layers lower than layer β.

[0131] Here, in a hierarchical coding method using RAHT, transform coefficients in a shallower upper layer tend to indicate low-frequency components of attribute information of three-dimensional points included in a target frame. In other words, the transform coefficients in the upper layer change little over time. Therefore, the coding device can improve coding efficiency by preferentially performing inter prediction on the transform coefficients in the upper layer. In this case, there is no need to transmit a code (raht_inter_node) indicating the prediction method (intra prediction or inter prediction) to be used, thereby reducing the amount of data in the bitstream.

[0132] In addition, the encoding device can select the prediction method that results in the smallest amount of code, between intra prediction and inter prediction, for transform coefficients in intermediate layers of intermediate depth (layers below layer α and above layer β), thereby improving encoding efficiency.

[0133] High-frequency components are dominant in transform coefficients in lower layers with greater depth. In other words, the transform coefficients in lower layers are likely to change significantly over time. Therefore, the encoding device can improve encoding efficiency by applying intra-prediction to the transform coefficients in lower layers, rather than inter-prediction.

[0134] In the above description, an example has been shown in which inter-priority prediction processing is applied to a higher layer, switching prediction processing is applied to a middle layer, and intra prediction is applied to a lower layer, but the processing applied to each layer is not limited to this. Any of the first, second, third, and fourth examples of the prediction method switching method described above, and the intra prediction and inter prediction methods may be combined.

[0135] Furthermore, for example, information indicating a prediction method to be applied to each layer may be stored in the bitstream. In this case, the decoding device performs prediction processing on the transform coefficients included in the layer using the prediction method indicated by the information. For example, the information may indicate whether to apply inter-priority prediction processing to the layer, whether to apply switching prediction processing, or whether to apply neither intra-prediction nor inter-prediction. This allows the encoding device to switch an appropriate prediction method for each layer, thereby improving encoding efficiency.

[0136] [Fifth Example of Predictive Decoding Process of Transform Coefficients] A fifth example of a method for switching prediction methods by a decoding device corresponding to the encoding device according to the fifth example above will be described. Fig. 20 is a flowchart of the fifth example of the decoding process of transform coefficients. The process shown in Fig. 20 is performed, for example, repeatedly for each node of an octree hierarchy included in a target frame to be decoded.

[0137] Next, the decoding device determines whether the depth of the octree hierarchy to which the target node belongs is the depth of the target of inter-priority prediction (S401). For example, the decoding device obtains (decodes) information indicating the depth of the target of inter-priority prediction from the bitstream, and makes the determination based on the obtained information.

[0138] If the depth to which the target node belongs is the depth to which inter-priority prediction is applied (Yes in S401), the decoding device applies inter-priority prediction processing to the target node (S402). Specifically, the decoding device determines whether a node at the same position as the target node exists in the reference frame (S411). If the decoding device determines that a node at the same position as the target node exists in the reference frame (Yes in S411), it applies inter-prediction to the target node and performs decoding (S412).

[0139] On the other hand, if the decoding device determines that there is no node at the same position as the target node in the reference frame (No in S411), the decoding device performs intra prediction processing (S413).

[0140] The details of the processes in steps S411 to S413 are the same as, for example, steps S202A, S203, and S205 shown in FIG.

[0141] On the other hand, if the depth to which the target node belongs is not the depth of the inter-priority prediction target (No in S401), the decoding device determines whether to apply switching prediction processing. Specifically, the decoding device determines whether the depth of the octree hierarchy to which the target node belongs is the depth of the switching prediction target (S403). For example, the decoding device acquires (decodes) information indicating the depth of the switching prediction target from the bitstream, and makes the determination based on the acquired information.

[0142] If the depth to which the target node belongs is the depth to be subjected to switching prediction (Yes in S403), the decoding device applies switching prediction processing to the target node (S404).

[0143] First, the decoding device decodes (obtains) information (rht_inter_node) indicating whether the transform coefficients for each node have been coded using intra prediction or inter prediction from the bitstream (S421).

[0144] Next, the decoding device refers to raht_inter_node and determines whether the target node should be decoded by intra prediction or inter prediction (S422). Specifically, if raht_inter_node=1 (Yes in S422), the decoding device performs inter prediction processing (S423). On the other hand, if raht_inter_node=0 (No in S422), the decoding device performs intra prediction processing (S424).

[0145] The details of the processes in steps S421 to S424 are the same as, for example, steps S251 to S254 shown in FIG.

[0146] If the depth to which the target node belongs is not the depth to be switched (No in S403), the decoding device applies intra prediction processing to the target node (S405). Note that the details of step S405 are the same as, for example, step S413.

[0147] After step S402, S404, or S405, the decoding device stores the transform coefficients of the target node in a memory for inter prediction (S406). Finally, the decoding device generates attribute information of the target node by performing inverse RAHT on the transform coefficients obtained in step S412, S413, S423, S424, or S405 (S407).

[0148] In this way, the decoding device can select the optimal prediction method according to the characteristics of the target node by applying an appropriate prediction method or combination of prediction methods depending on the depth of the node. This can improve coding efficiency. Furthermore, the bitstream with improved coding efficiency generated by the above-mentioned encoding device can be properly decoded.

[0149] In the above description, an example has been shown in which inter-priority prediction processing is applied to a higher layer, switching prediction processing is applied to a middle layer, and intra prediction is applied to a lower layer, but the processing applied to each layer is not limited to this. Any of the first, second, third, and fourth examples of the prediction method switching method described above, and the intra prediction and inter prediction methods may be combined.

[0150] Furthermore, information indicating a prediction method to be applied to each layer may be stored in the bitstream. In this case, the decoding device performs prediction processing on the transform coefficients included in the layer using the prediction method indicated by the information. For example, the information may indicate whether to apply inter-priority prediction processing to the layer, whether to apply switching prediction processing, or whether to apply neither intra prediction nor inter prediction. This allows the decoding device to switch an appropriate prediction method for each layer, thereby improving coding efficiency.

[0151] The plurality of methods may include a plurality of inter prediction methods. For example, the plurality of inter prediction methods may include an inter prediction method that applies motion compensation (or positional correction) and an inter prediction method that does not apply motion compensation (or positional correction).

[0152] For example, inter prediction without motion compensation may be used for a higher layer that is less affected by motion, and inter prediction with motion compensation may be used for a lower layer that is affected by motion, which may improve coding efficiency.

[0153] 18 and 20 have described an example in which, as a method for determining the method to be applied, a determination of inter-priority prediction method is made first, and then a determination of switching prediction method is made, but the order of determination is not limited to this. Also, even when multiple methods other than those described above are used, the determinations may be made in any order.

[0154] Note that, in the above-mentioned S107 of Figures 3 and 9, S206 of Figures 7 and 11, S135 of Figure 12, S234 of Figure 14, S159 of Figure 16, S255 of Figure 17, S307 of Figure 18, and S406 of Figure 20, instead of transform coefficients, attribute information obtained by inversely transforming the transform coefficients may be stored in memory. In this case, in predicting subsequent transform coefficients, transform coefficients may be generated by transforming the attribute information stored in memory, and these transform coefficients may be used for prediction. Note that, motion compensation may be performed on the attribute information stored in memory, and transform coefficients may be generated by transforming the attribute information after motion compensation, and these transform coefficients may be used for prediction.

[0155] [Syntax Example] Fig. 21 is a diagram showing an example of the syntax of attribute data (attribute_data). The attribute data is coded data generated by coding attribute information. As shown in Fig. 21, the attribute data includes ZeroCnt, raht_inter_coeff[i+j], and attribute_value[i].

[0156] ZeroCnt indicates the number of consecutive zeros in a quantized transform coefficient. ZeroCnt may be binarized and arithmetically coded.

[0157] raht_inter_coeff[i+j] indicates whether the (i+j)th transform coefficient, coeff, is coded using intra prediction or inter prediction. For example, when the value of raht_inter_coeff[i+j] is 1, it indicates that the (i+j)th coeff is coded using inter prediction. When the value of raht_inter_coeff[i+j] is 0, it indicates that the (i+j)th coeff is coded using intra prediction.

[0158] As a result, by decoding raht_inter_coeff[i+j], the decoding device can determine whether to decode the (i+j)th coeff using intra prediction or inter prediction, and can appropriately decode the bitstream. Note that the value of raht_inter_coeff may be entropy coded and added to the header. For example, raht_inter_coeff may be binarized and arithmetically coded.

[0159] Furthermore, the encoding device may arithmetically encode raht_inter_coeff while adaptively switching the context to be used. For example, the encoding device may switch the context to be used depending on the value of raht_inter_coeff coded before the raht_inter_coeff to be coded. More specifically, the encoding device may select context 0 when the value of raht_inter_coeff coded immediately before the raht_inter_coeff to be coded is 0, that is, the immediately preceding coeff was coded using intra prediction, and may select context 1 when the value of raht_inter_coeff coded immediately before the raht_inter_coeff to be coded is 1, that is, the immediately preceding coeff was coded using inter prediction. As a result, in cases where intra prediction or inter prediction occurs consecutively, the efficiency of arithmetic coding of raht_inter_coeff is improved and the amount of code can be reduced. Note that the coding device may code raht_inter_coeff at a fixed length in order to reduce the amount of processing.

[0160] Note that, here, an example has been shown in which information (raht_inter_coeff) indicating whether intra prediction or inter prediction has been used for each coeff is added to the bitstream, but this is not necessarily limited to this. For example, raht_inter_node, which is information indicating whether intra prediction or inter prediction has been used for each target node to which a target coeff belongs, may be added to the bitstream, and this information may be shared by multiple coeffs belonging to the target node. More specifically, when a target node includes three coeffs, if the value of raht_inter_node added to the target node is 1, the decoding device determines that all three coeffs belonging to the target node have been coded using inter prediction. Also, if the value of raht_inter_node added to the target node is 0, the decoding device determines that all three coeffs belonging to the target node have been coded using intra prediction. In this way, by sharing raht_inter_node among multiple coeffs included in the same target node, it is possible to reduce the amount of coding in the bitstream while appropriately applying intra prediction and inter prediction, thereby improving coding efficiency.

[0161] Note that the encoding device may perform arithmetic coding of raht_inter_node while adaptively switching the context to be used. For example, the encoding device may switch the context to be used depending on the value of raht_inter_node coded before the raht_inter_node to be coded. More specifically, the encoding device may select context 0 when the value of raht_inter_node coded immediately before the raht_inter_node to be coded is 0, that is, the immediately preceding node is coded using intra prediction, and may select context 1 when the value of raht_inter_node coded immediately before the raht_inter_node to be coded is 1, that is, the immediately preceding node is coded using inter prediction. This improves the efficiency of arithmetic coding of raht_inter_node in cases where intra prediction or inter prediction occurs consecutively, thereby reducing the amount of code. The encoding device may encode raht_inter_node with a fixed length in order to reduce the amount of processing.

[0162] In addition, the encoding device may add raht_inter_coeff or raht_inter_node to the bitstream when the target coeff or target node is included in the range (depth) of the inter prediction target or the switching prediction target. For example, when the depth of the target node to which the target coeff belongs is the depth of the inter prediction target or the switching prediction target, raht_inter_coeff or raht_inter_node is added to the bitstream. This allows the decoding device to be notified whether the target coeff was coded using intra prediction or inter prediction. On the other hand, when the target node to which the target coeff belongs is not included in the range of any of the inter prediction target, the switching prediction target, and the inter priority prediction target, the encoding device does not need to add raht_inter_coeff or raht_inter_node to the bitstream. In this case, the decoding device may consider raht_inter_coeff=0 or raht_inter_node=0 and decode the target coeff using intra prediction.

[0163] In this way, when the target node to which the target coeff belongs is included in the range of the inter prediction target or the switch prediction target, the encoding device adds raht_inter_coeff or raht_inter_node to the bitstream. This allows encoding to be performed while switching between intra prediction and inter prediction for each target coeff while suppressing the code amount of the bitstream, thereby improving encoding efficiency.

[0164] attribute_value[i] indicates the value of the i-th coeff (transform coefficient). Note that attribute_value may be composed of multiple components. For example, attribute_value may include a sign bit indicating whether the transform coefficient is positive or negative, and a coefficient value indicating the absolute value of the transform coefficient.

[0165] Alternatively, the attribute_value may be entropy coded and added to the header. For example, the attribute_value may be binarized and arithmetically coded. Alternatively, to reduce the amount of processing, the attribute_value may be coded with a fixed length.

[0166] In addition, when each bit after binarization is arithmetically coded, the coding device may switch coding tables (or contexts) for each bit. This can improve coding efficiency. For example, the context for arithmetically coding attribute_value[i] may be switched depending on the value of raht_inter_coeff[i] of the i-th coeff. Specifically, raht_inter_coeff[i] = 0, i.e., a context for intra prediction, and raht_inter_coeff[i] = 1, i.e., a context for inter prediction, may be prepared, and the context used in arithmetic coding may be switched depending on whether intra prediction or inter prediction is used for the target node. This allows appropriate contexts to be used for intra prediction and inter prediction, even when the prediction accuracy differs between intra prediction and inter prediction and the tendency of transform coefficients to occur differs, thereby improving coding efficiency.

[0167] It should be noted that the context switching method for attribute_value is not limited to the above method. For example, the context may be switched based on whether the depth of the target node to which coeff belongs is included in the range (depth) of inter-prediction targets. For example, when the depth of the target node to which coeff belongs is included in the range of inter-prediction targets, attribute_value is a transform coefficient generated by intra-prediction or a transform coefficient generated by inter-prediction. On the other hand, when the depth of the target node to which coeff belongs is not included in the range of inter-prediction targets, attribute_value is a transform coefficient generated by intra-prediction. In this way, depending on whether the depth of the target node is included in the range of inter-prediction targets, the value of attribute_value may have different generation tendencies, and therefore, by using different contexts in each case, coding efficiency can be improved.

[0168] Furthermore, the encoding device may switch the context used for arithmetic coding of attribute_value[i] according to the value of raht_inter_node of the node to which the i-th coeff belongs. This allows appropriate contexts to be used for intra prediction and inter prediction even when the prediction accuracy differs between intra prediction and inter prediction and the tendency of transform coefficients to occur differs. Therefore, encoding efficiency can be improved.

[0169] Although the context switching in arithmetic coding by the encoding device has been described above, a similar technique can also be applied to context switching in arithmetic decoding by the decoding device.

[0170] 22 is a diagram showing an example of APS syntax. APS (Attribute Parameter Set) is control information (also called a parameter set or metadata) included in a bitstream, and is control information related to encoding of attribute information. For example, APS is control information common to multiple frames.

[0171] As shown in FIG. 22, the APS includes inter_raht_prediction_enabled, depth, and raht_inter_priority_enable.

[0172] inter_raht_prediction_enable is a flag indicating whether inter prediction is applied to attribute information (transform coefficients after RAHT). In other words, this flag indicates whether the inter prediction method is enabled. Note that inter_raht_prediction_enable is stored in the APS when the RAHT method is selected as the coding method for the attribute information, and may not be stored in the APS otherwise. Also, inter_raht_prediction_enable is stored in the APS when the RAHT method is selected and it is selected that predictive coding is to be applied in the RAHT method, and may not be stored in the APS otherwise. This allows the amount of code in the header to be reduced.

[0173] When inter_raht_prediction_enabled indicates that inter prediction is to be applied, the APS includes depth and raht_inter_priority_enable.

[0174] Depth is information indicating the depth to which inter prediction is applied (range of inter prediction targets). For example, it may be specified that inter prediction is always applied to the highest layer, and depth may indicate the depth -1 to which inter prediction is applied. For example, inter prediction is applied to layers above (or equal to or greater than) the depth indicated by depth (or depth -1), and inter prediction is not applied to layers below (or less than) the depth indicated by depth (or depth -1).

[0175] raht_inter_priority_enable is information indicating whether inter prediction is prioritized (whether inter prediction or intra prediction is prioritized). When the value of raht_inter_priority_enable is 1, inter prediction is prioritized as in the first example, and when the value of raht_inter_priority_enable is 0, intra prediction is prioritized as in the third example. As a result, for example, for a scene with fast motion, intra prediction can be prioritized by setting raht_inter_priority_enable to a value of 0, and otherwise inter prediction can be prioritized by setting raht_inter_priority_enable to a value of 1. This allows the encoding device to switch the prediction method to be used to a prediction method appropriate for the scene, thereby improving encoding efficiency.

[0176] Furthermore, the encoding device can reduce the amount of bits by adding raht_inter_priority_enable to the bitstream when inter_raht_prediction_enabled=1, that is, when inter prediction is applied.

[0177] Fig. 23 is a diagram showing another example of the syntax of an APS. The APS shown in Fig. 23 includes depth_u and depth_l instead of depth, as compared to the APS shown in Fig. 22. Furthermore, the APS shown in Fig. 23 has raht_inter_prediction_type added to the APS shown in Fig. 22.

[0178] Fig. 24 is a diagram for explaining depth_u and depth_l. Depth_u is information indicating the depth (range) to which inter-priority prediction processing is applied in the processing shown in Fig. 18 and Fig. 20. For example, as shown in Fig. 24, the range to which inter-priority prediction processing is applied is the range from the highest layer to the depth (layer) indicated by depth_u. For example, it may be specified that inter-priority prediction processing is always applied to the highest layer, and depth_u may indicate the depth -1 to which inter-priority prediction processing is applied.

[0179] Depth_l is information indicating the depth (range) to which the switching prediction process is applied in the process shown in Figures 18 and 20. For example, as shown in Figure 24, the range to which the inter-priority prediction process is applied is the range between the depth (layer) indicated by depth_u and the depth (layer) indicated by depth_l. Depth_l may indicate the depth -1 to which the switching prediction process is applied. Depth_l may be determined to be a layer lower than depth_u. Furthermore, depth_l may indicate a difference value between the depth to which the switching prediction process is applied and the depth to which the inter-priority prediction process indicated by depth_u is applied.

[0180] Furthermore, depth_u may indicate the depth from a higher layer (the difference between the top layer and the bottom layer of the layers to which inter-priority prediction processing is applied), and depth_l may indicate the depth from a lower layer (the difference between the bottom layer and the bottom layer of the layers to which switching prediction processing is applied). In this case, the decoding device may determine that a conformance violation has occurred when actual depth-depth_l<(<=)depth_u.

[0181] It is not essential to transmit information indicating the depth (range) to which the switching prediction process is applied to the decoding device, and a depth (range) previously agreed upon between the encoding device and the decoding device may be used without transmitting this information.

[0182] When inter_raht_prediction_enabled indicates that an inter prediction scheme is to be applied, the APS includes raht_inter_prediction_type, which is a flag indicating whether the first inter prediction or the second inter prediction is to be applied. Here, the first inter prediction is, for example, an inter prediction scheme that does not apply motion compensation, and the second inter prediction is, for example, an inter prediction scheme that applies motion compensation.

[0183] [Modifications] In any of the first to fifth examples above, the encoding device may always determine the conditions for performing intra-prediction shown in Figures 4 and 5, regardless of the depth of the target node or whether inter-prediction is applied. Alternatively, the encoding device may calculate the number of neighboring nodes required for intra-prediction regardless of the prediction mode applied. Here, when processing switches from a layer to which inter-prediction is applied to a layer to which intra-prediction is applied, the number of neighboring nodes is required for intra-prediction processing. In contrast, by calculating the number of neighboring nodes even when inter-prediction is applied, intra-prediction processing can be applied immediately when switching to intra-prediction, thereby improving encoding efficiency.

[0184] Furthermore, the encoding device may initialize control of determining the conditions for performing intra-prediction when the depth of the target node is the depth targeted for inter-prediction, or when inter-prediction is applied and intra-prediction is not performed. For example, this initialization allows the conditions for performing intra-prediction to be applied immediately if inter-prediction is not subsequently performed. More specifically, in determining the conditions for performing intra-prediction shown in FIG. 5 , the encoding device initializes the number of neighboring nodes to a value equal to or greater than a threshold. This ensures that the conditions for performing intra-prediction are met when intra-prediction is resumed. Therefore, intra-prediction can be applied immediately when switching intra-prediction, thereby improving encoding efficiency.

[0185] Furthermore, the encoding device may read attribute information of the parent node used in intra prediction from the reference memory regardless of the applied prediction mode, thereby reducing the processing time when intra prediction is selected.

[0186] In intra prediction, the encoding device may first calculate a predicted value of the attribute information of the target node using attribute information of neighboring nodes, and then apply RAHT transform to calculate a predicted transform coefficient. On the other hand, in inter prediction, the encoding device may store the predicted transform coefficient of the reference frame in advance in a reference buffer and read out the value to use as the predicted transform coefficient. In other words, the calculation method of the predicted transform coefficient may differ between intra prediction and inter prediction. As a result, when inter prediction is selected, the encoding device can reduce the processing amount and overall processing time by using the predicted transform coefficient of the reference frame as is, compared to intra prediction in which RAHT transform is applied to the predicted value.

[0187] Furthermore, the information stored in the reference buffer may differ between intra prediction and inter prediction. For example, the reference buffer for intra prediction may store attribute information of neighboring nodes, and the reference buffer for inter prediction may store transform coefficients of attribute information of the reference frame. By switching the information stored in the reference buffer depending on the prediction method in this way, the processing efficiency of each prediction method can be improved, thereby reducing the amount of processing.

[0188] For example, in intra prediction, the encoding device calculates a predicted value from attribute information of a neighboring node, performs RAHT transformation on the predicted value to calculate a predicted transform coefficient, calculates a difference value (prediction residual) by subtracting the predicted transform coefficient from the transform coefficient of the target node, and encodes (e.g., entropy encodes) the difference value. Also, in inter prediction, the encoding device reads out the predicted transform coefficient at the same position from a reference frame, calculates a difference value by subtracting the predicted transform coefficient from the transform coefficient of the target node, and encodes (e.g., entropy encodes) the difference value.

[0189] The encoding device may perform inter prediction on transform coefficients when intra prediction is not possible, and vice versa, thereby increasing the number of nodes that can be predicted for a target node, thereby improving encoding efficiency.

[0190] The following may be used as conditions for implementing inter prediction (methods for selecting a prediction method), for example: The encoding device may select a prediction method (inter prediction or intra prediction) based on the distance between the position of the target node and the position of the reference node. For example, if the distance is closer than a predetermined threshold, the encoding device may determine that the accuracy of inter prediction is higher and select inter prediction. If the distance is equal to or greater than the threshold, the encoding device may determine that the accuracy of intra prediction is higher than inter prediction and select intra prediction. In this way, the encoding device can improve encoding efficiency by appropriately switching between inter prediction and intra prediction depending on the distance between the target node and the reference node.

[0191] For example, the encoding device may select a prediction method based on whether the patterns of neighboring nodes of the target node and neighboring nodes of the reference node match or do not match. For example, the greater the pattern match, the greater the effect of inter prediction is expected in terms of similarity. Therefore, the encoding device selects inter prediction if the patterns match, and selects intra prediction if they do not. This can improve encoding efficiency.

[0192] For example, the encoding device may select a prediction method based on whether the movement speed or movement direction of the target node and the reference node match or do not match. When the movement speed or movement direction matches, inter prediction is expected to be more effective. Therefore, the encoding device selects inter prediction when the movement speed or movement direction matches to a certain degree or more (when the difference is less than a threshold), and selects intra prediction otherwise. This can improve encoding efficiency.

[0193] For example, the encoding device may select a prediction method based on the correlation between the attributes of the target node and the reference node. When the correlation between the attributes is high, inter-prediction is expected to be more effective. Therefore, the encoding device selects inter-prediction when the correlation between the attributes of the target node and the reference node is equal to or greater than a predetermined threshold, and selects intra-prediction otherwise. This improves encoding efficiency.

[0194] For example, the encoding device may select a prediction method based on the density of three-dimensional points included in the processing unit (e.g., sequence, frame, slice, etc.) to which the target node or reference node belongs. When the density is high, the distance between adjacent nodes is small, and intra prediction is expected to be effective. Therefore, the encoding device may determine not to perform inter prediction (to perform intra prediction) when the density is high. Note that the density can be determined based on, for example, the number of nodes per unit space, the distance between the target node and neighboring nodes, or the pattern of neighboring nodes.

[0195] It should be noted that the above-described conditions may be combined. For example, a predetermined prediction method (inter prediction or intra prediction) may be selected when all of two or more conditions are satisfied, or a predetermined prediction method (inter prediction or intra prediction) may be selected when at least one of two or more conditions is satisfied.

[0196] Furthermore, the execution condition to be used may be selected from a plurality of execution conditions for each processing unit, where the processing unit is, for example, a node, a slice, a frame, or a sequence.

[0197] The plurality of execution conditions may include a mode in which intra prediction is unconditionally performed or a mode in which intra prediction is unconditionally not performed. The plurality of execution conditions may also include a mode in which inter prediction is unconditionally performed or a mode in which inter prediction is unconditionally not performed.

[0198] Similarly, a condition for implementing the third prediction method other than the intra prediction method and the inter prediction method may be set, and the encoding device may implement the third prediction method based on the implementation condition for one of the three prediction methods, and if the prediction method is not implemented, may implement one of the remaining two prediction methods.Also, if neither of the two prediction methods is implemented, the encoding device may implement the remaining prediction method.

[0199] Although the operation of the encoding device has been described as an example here, the same operation may also be performed in the decoding device.

[0200] The encoding device may also determine the priority between intra-prediction and inter-prediction, which is expected to enable the encoding device to select the optimal prediction method depending on the sequence or situation, thereby improving encoding efficiency.

[0201] The following conditions, for example, can be used as conditions for whether to prioritize intra prediction (whether to prioritize intra prediction or inter prediction). For example, the encoding device may determine the prediction method (intra prediction or inter prediction) to be prioritized based on the hierarchical layer of the RAHT to which the target node belongs. Here, the lower the hierarchical layer, the narrower the spatial range, and therefore the more likely it is that intra prediction is more efficient. Furthermore, the higher the hierarchical layer, the more difficult intra prediction is, and therefore the more likely it is that inter prediction is more effective. Therefore, the encoding device may prioritize intra prediction for nodes belonging to lower hierarchical layers, and prioritize inter prediction for nodes belonging to higher hierarchical layers.

[0202] For example, the encoding device may determine a preferred prediction method based on the density of three-dimensional points included in the target processing unit or the reference processing unit. Here, the processing unit may be, for example, a sequence, a frame, or a slice. Intra prediction is likely to be more efficient for processing units with dense point groups, while inter prediction is likely to be more efficient for processing units with sparse point groups. Therefore, the encoding device may prioritize intra prediction when the density of the processing unit is equal to or greater than a predetermined threshold, and prioritize inter prediction when the density of the processing unit is less than the threshold.

[0203] For example, the encoding device may determine a preferred prediction method based on the difference in motion, movement speed, or movement direction between the target processing unit and the reference processing unit. For example, when the motion between the two is small, inter prediction is likely to be effective, while when the motion is large, intra prediction is likely to be efficient. This tendency also applies to movement speed and movement direction. Therefore, the encoding device may prioritize inter prediction when the difference in motion, movement speed, or movement direction is less than a predetermined threshold, and prioritize intra prediction when the difference in motion, movement speed, or movement direction is equal to or greater than the threshold.

[0204] For example, the encoding device may determine a prioritized prediction method based on the correlation between the attributes of the target processing unit and the reference processing unit. When the absolute amount of correlation is large, inter prediction is likely to be more efficient, while when the absolute amount of correlation is small, intra prediction is likely to be more efficient. Therefore, the encoding device may prioritize inter prediction when the absolute amount of correlation is equal to or greater than a predetermined threshold, and prioritize intra prediction when the absolute amount of correlation is less than the threshold.

[0205] For example, the encoding device may determine a preferred prediction method based on a setting value stored in control information such as an SPS included in the bitstream. This allows the encoding device to select a prediction method appropriate for the sequence. The SPS (Sequence Parameter Set) is control information (parameter set) common to multiple frames.

[0206] The decoding device may also determine a preferred prediction method based on information such as a flag included in the bitstream, thereby selecting a prediction method appropriate for a sequence or situation. The information may also be stored for each node, slice, frame, or sequence.

[0207] It should be noted that the above-described multiple conditions may be combined. For example, a predetermined prediction method (inter prediction or intra prediction) may be prioritized when all of two or more conditions are satisfied, or a predetermined prediction method (inter prediction or intra prediction) may be prioritized when at least one of two or more conditions is satisfied.

[0208] Furthermore, a condition to be used may be selected from a plurality of conditions for each processing unit, where the processing unit is, for example, a node, a slice, a frame, or a sequence.

[0209] The plurality of conditions may include a mode in which intra prediction is unconditionally prioritized, or a mode in which intra prediction is not unconditionally prioritized, and may also include a mode in which inter prediction is unconditionally prioritized, or a mode in which inter prediction is not unconditionally prioritized.

[0210] Similarly, a condition for priority of a third prediction method other than the priority of intra prediction and the priority of inter prediction may be set, and the encoding device may select a prediction method to be prioritized based on the condition for priority of one of the three prediction methods, and if that prediction method is not prioritized, prioritize one of the remaining two prediction methods.Also, if neither of the two prediction methods is prioritized, the encoding device may prioritize the remaining prediction method.

[0211] The third prediction method may be a method that does not perform prediction. That is, the third prediction method may be a method that does not apply either inter prediction or intra prediction. Furthermore, the third prediction method may be applied as an additional method to any of the methods in the first to fifth examples.

[0212] When the third prediction method is used, raht_inter_node may take three values: 0, 1, or 2. raht_inter_node may also include two flags: raht_inter_node_1 and raht_inter_node_2. raht_inter_node_1 and raht_inter_node_2 are each, for example, a flag of 1 that takes two values: 0 or 1.

[0213] For example, raht_inter_node_1 = 0 indicates intra prediction or no prediction (neither intra prediction nor inter prediction is applied), raht_inter_node_1 = 1 indicates inter prediction, raht_inter_node_2 = 0 indicates no prediction, and raht_inter_node_2 = 1 indicates intra prediction. Also, the syntax shown in Fig. 25 may be used. For example, the syntax shown in Fig. 25 may be included in the syntax of attribute data (attribute_data).

[0214] As shown in Figure 25, when raht_inter_node_1 = 0, raht_inter_node_2 is included in the bitstream, and when raht_inter_node_1 = 1, raht_inter_node_2 is not included in the bitstream. That is, when inter prediction is applied, only raht_inter_node_1 = 1 is stored in the bitstream, and raht_inter_node_2 is not stored in the bitstream. When intra prediction is applied, raht_inter_node_1 = 0 and raht_inter_node_2 = 1 are stored in the bitstream. When no prediction is applied, raht_inter_node_1 = 0 and raht_inter_node_2 = 0 are stored in the bitstream.

[0215] In this way, depending on the logic of one of the two flags, the other flag may be stored in the bitstream.

[0216] The above-described correspondence relationship between the values ​​of raht_inter_node_1 and raht_inter_node_2 and the prediction methods is merely an example, and other relationships may be used.Furthermore, the condition for the existence of the other flag may also be other than those described above.

[0217] Furthermore, a method may be switched between a method using only the first and second prediction methods without using the third prediction method, and a method using all of the first to third prediction methods. In this case, a signal for performing the switching may be stored in an SPS, an APS, or an ADUH (Attribute Data Unit Header). The ADUH is an ADU header (control information) of an ADU (Attribute Data Unit). The ADU is a data unit of coded data of attribute information.

[0218] Although the operation of the encoding device has been described as an example here, the same operation may also be performed in the decoding device.

[0219] Furthermore, although the above describes an example in which prediction is performed on transform coefficients generated by RAHT, a similar method may be applied to coefficients generated by a transform method other than RAHT. If coefficients generated by a transform method other than RAHT are correlated, there is a possibility that coding efficiency will be improved. The transform method other than RAHT may be, for example, a hierarchical transform using a dyad tree, such as RAHT. Furthermore, the transform method other than RAHT is not limited to a hierarchical transform, and may be any method that obtains a value for each component of attribute information. For example, the transform method other than RAHT may be principal component analysis.

[0220] [Summary] A decoding device (3D data decoding device) according to an embodiment performs the processing shown in Fig. 26. The decoding device prioritizes inter prediction over intra prediction in a first prediction process when the depth of a target node storing a 3D point is less than a first threshold (S501), uses a prediction method indicated by control information included in the bitstream in a second prediction process when the depth is less than a second threshold that is greater than the first threshold and is equal to or less than the first threshold (S502), and uses intra prediction in a third prediction process when the depth is equal to or less than the second threshold (S503).

[0221] According to this, the encoding device generates a bitstream with improved encoding efficiency by using a prediction method suitable for the depth of each node, and the decoding device can appropriately decode the bitstream.

[0222] For example, in at least one of the first prediction process, the second prediction process, and the third prediction process, neither intra prediction nor inter prediction is applied according to a predetermined condition. This allows the encoding device to encode some nodes according to the predetermined condition without using a prediction process, thereby generating a bitstream with improved encoding efficiency.

[0223] For example, the decoding device further determines a prediction process to be used for the target node by using first information (e.g., depth_u and depth_l) indicating a first threshold and a second threshold included in the bitstream. In this way, the decoding device can determine a prediction process to be used for the target node by using the first information.

[0224] For example, the first information includes second information (e.g., depth_u) indicating the depth from the shallowest depth at which the first threshold is located, and third information (e.g., depth_l) indicating the depth from the deepest depth at which the second threshold is located, which can reduce the amount of data in the first information.

[0225] For example, the control information includes a first flag (e.g., raht_inter_node_1) for a target node for which the second prediction process is used, and when the first flag indicates a first value (e.g., 0), the control information further includes a second flag (e.g., raht_inter_node_2) for the target node, and when the second flag indicates a third value (e.g., 1), intra prediction is used in the prediction process of the target node, when the second flag indicates a fourth value (e.g., 0), neither inter prediction nor intra prediction is used in the prediction process of the target node, and when the first flag indicates a second value (e.g., 1), the second flag is not included in the bitstream, and inter prediction is used in the prediction process of the target node. This allows for a reduction in the amount of data in the control information and a reduction in the amount of processing in the decoding device when inter prediction is frequently used.

[0226] 27 is a block diagram of the decoding device 10. For example, the decoding device 10 includes a processor 11 and a memory 12, and the processor 11 uses the memory 12 to perform the above-described processing.

[0227] Moreover, the encoding device (three-dimensional data encoding device) according to the embodiment performs the processing shown in Fig. 28. In the first prediction process when the depth of the target node storing the three-dimensional point is less than a first threshold, the encoding device prioritizes inter prediction over intra prediction (S511), in the second prediction process when the depth is less than a second threshold that is greater than the first threshold and is equal to or less than the first threshold, control information indicating the prediction method used is stored in the bitstream (S512), and in the third prediction process when the depth is equal to or less than the second threshold, intra prediction is used (S513).

[0228] This allows the encoding device to generate a bitstream with improved encoding efficiency by using a prediction method suited to the depth of each node.

[0229] The encoding device may perform the same process as the decoding device. For example, the encoding device may perform a process in which the decoding in the decoding device is replaced with encoding.

[0230] 29 is a block diagram of the encoding device 20. For example, the encoding device 20 includes a processor 21 and a memory 22, and the processor 21 uses the memory 22 to perform the above-described processing.

[0231] The above describes the encoding device (three-dimensional data encoding device) and decoding device (three-dimensional data decoding device) according to the embodiments and modifications of the present disclosure, but the present disclosure is not limited to these embodiments.

[0232] For example, in the above-described embodiment and modified examples, the target of the switched prediction process was a node, but the target of the switched prediction process may be another data unit that stores three-dimensional data. Furthermore, in the above-described embodiment and modified examples, the criterion for switching the prediction process was depth, but the criterion for switching the prediction process may be anything related to the position of the data unit. For example, the branching level of a tree structure or the detail of the data may be used instead of depth.

[0233] Furthermore, each processing unit included in the encoding device, decoding device, etc. according to the above-described embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0234] Furthermore, the integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI may also be used.

[0235] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0236] In addition, the present disclosure may be realized as an encoding method (three-dimensional data encoding method) or a decoding method (three-dimensional data decoding method) executed by an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device), etc.

[0237] The present disclosure may also be realized as a program that causes a computer, processor, or device to execute the encoding method or decoding method. The present disclosure may also be realized as a bitstream generated by the encoding method. The present disclosure may also be realized as a recording medium on which the program or the bitstream is recorded. For example, the present disclosure may also be realized as a non-transitory computer-readable recording medium on which the program or the bitstream is recorded.

[0238] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0239] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0240] While the encoding device and decoding device according to one or more aspects have been described based on the embodiments, the present disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the present embodiments and configurations constructed by combining components of different embodiments may also be included within the scope of one or more aspects.

[0241] The present disclosure is applicable to encoding devices and decoding devices.

[0242] 10 Decoding device 11, 21 Processor 12, 22 Memory 20 Encoding device

Claims

1. In the first prediction process, when the depth of the target node storing the three-dimensional points is less than the first threshold, interpretation is prioritized over intraprediction. In the second prediction process when the depth is less than the second threshold and greater than the first threshold, the prediction method indicated by the control information included in the bitstream is used. In the third prediction process when the depth is greater than the second threshold, the intra prediction is used. Decryption method.

2. In at least one of the first prediction process, the second prediction process, and the third prediction process, neither the intra prediction nor the inter prediction is applied depending on predetermined conditions. The decoding method according to claim 1.

3. The aforementioned decoding method further, The prediction process to be used for the target node is determined using the first information, which includes the first threshold and the second threshold, contained in the bitstream. The decoding method according to claim 1.

4. The first piece of information mentioned above is, The first threshold is a second piece of information indicating the depth from the shallowest depth, The second threshold includes a third piece of information indicating which depth it is relative to the deepest depth. The decoding method according to claim 3.

5. The control information includes a first flag for the target node on which the second prediction process is used. If the first flag indicates a first value, The control information further includes a second flag for the target node, If the second flag indicates a third value, the intra prediction is used in the prediction process for the target node. If the second flag indicates a fourth value, neither the interpretation nor the intrapretation is used in the prediction process of the target node. If the first flag indicates the second value, The second flag is not included in the bitstream. The prediction process for the target node uses the aforementioned interpretation. The decoding method according to claim 1.

6. Processor and Equipped with memory, The processor uses the memory to: In the first prediction process, when the depth of the target node storing the three-dimensional points is less than the first threshold, interpretation is prioritized over intraprediction. In the second prediction process when the depth is less than the second threshold and greater than the first threshold, the prediction method indicated by the control information included in the bitstream is used. In the third prediction process when the depth is greater than the second threshold, the intra prediction is used. Decoding device.

7. Processor and Equipped with memory, The processor uses the memory to: In the first prediction process, when the depth of the target node storing the three-dimensional points is less than the first threshold, interpretation is prioritized over intraprediction. In the second prediction process where the depth is less than the second threshold and greater than the first threshold, control information indicating the prediction method used is stored in the bitstream. In the third prediction process when the depth is greater than the second threshold, the intra prediction is used. Encoding device.

Images