Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

Abstract

This invention provides a three-dimensional data encoding method that can improve encoding efficiency. [Solution] A three-dimensional data encoding method for encoding three-dimensional points having a prediction tree structure, comprising encoding the node position based on a method corresponding to one of one or more selectable methods used to predict the node position, generating a bitstream containing mode information indicating the method used for the node (S9784), wherein the one or more selectable methods are modified based on the depth of the node in the prediction tree structure.

Description

[Technical Field]

[0001] This disclosure relates to a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, and a three-dimensional data decoding device. [Background technology]

[0002] In the future, devices and services utilizing three-dimensional data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, monitoring, infrastructure inspection, and video distribution. Three-dimensional data can be acquired in various ways, such as using distance sensors like rangefinders, stereo cameras, or combinations of multiple monocular cameras.

[0003] One method of representing three-dimensional data is called a point cloud, which represents the shape of a three-dimensional structure using a cloud of points in three-dimensional space. In a point cloud, the position and color of the points are stored. Point clouds are expected to become the mainstream method of representing three-dimensional data, but point clouds are extremely large in size. Therefore, in the storage or transmission of three-dimensional data, data compression through encoding is essential, just as with two-dimensional moving images (for example, MPEG-4 AVC or HEVC, which are standardized by MPEG).

[0004] Furthermore, point cloud compression is partially supported by publicly available libraries (such as the Point Cloud Library) that handle point cloud-related processing.

[0005] Furthermore, there is a known technique for searching for and displaying facilities located around a vehicle using three-dimensional map data (see, for example, Patent Document 1). [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] International Publication No. 2014 / 020663 [Overview of the project] [Problems that the invention aims to solve]

[0007] It is desirable to be able to improve coding efficiency in the encoding of three-dimensional data.

[0008] The purpose of this disclosure is to provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency. [Means for solving the problem]

[0009] A three-dimensional data encoding method according to one aspect of the present disclosure is a three-dimensional data encoding method for encoding three-dimensional points having a prediction tree structure, comprising encoding the location of the node based on a method corresponding to one of one or more selectable methods used to predict the location of the node, generating a bitstream containing mode information indicating the method used for the node, wherein the one or more selectable methods are modified based on the depth of the node in the prediction tree structure.

[0010] A three-dimensional data decoding method according to one aspect of the present disclosure is a three-dimensional data decoding method for decoding three-dimensional points having a prediction tree structure, wherein mode information indicating a method used to predict the position of a node in the prediction tree structure is obtained, and the position of the node is decoded based on the method indicated by the mode information, wherein the method indicated by the mode information corresponds to one of one or more selectable methods that are modified based on the depth of the node in the prediction tree structure.

[0011] A three-dimensional data encoding method according to an aspect of the present disclosure is a three-dimensional data encoding method for encoding a plurality of three-dimensional points having a hierarchical structure, and uses second position information of one or more second three-dimensional points around a first three-dimensional point to set one prediction mode out of two or more prediction modes for calculating a predicted value of the first position information of the first three-dimensional point, calculates the predicted value of the set prediction mode, calculates a prediction residual which is a difference between the first position information and the calculated predicted value, generates a first bit stream including the set prediction mode and the prediction residual, and in the setting, sets the prediction mode based on the depth of the hierarchical structure of the first three-dimensional point.

[0012] A three-dimensional data decoding method according to an aspect of the present disclosure is a three-dimensional data decoding method for decoding a plurality of three-dimensional points having a hierarchical structure, and obtains a first bit stream including an encoded prediction mode of a first three-dimensional point among the plurality of three-dimensional points and an encoded prediction residual, decodes a prediction mode value indicating the encoded prediction mode and the encoded prediction residual, calculates a predicted value of a prediction mode indicated by the prediction mode value obtained by decoding, calculates the first position information of the first three-dimensional point by adding the predicted value and the prediction residual obtained by decoding, and the encoded prediction mode included in the first bit stream is a prediction mode set based on the depth of the hierarchical structure of the first three-dimensional point.

Advantages of the Invention

[0013] The present disclosure can provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency.

Brief Description of the Drawings

[0014] [Figure 1] FIG. 1 is a diagram showing the configuration of a three-dimensional data encoding / decoding system according to Embodiment 1. [Figure 2] FIG. 2 is a diagram showing a configuration example of point cloud data according to Embodiment 1. [Figure 3]FIG. 3 is a diagram showing a configuration example of a data file in which point group data information according to Embodiment 1 is described. [Figure 4] FIG. 4 is a diagram showing the types of point group data according to Embodiment 1. [Figure 5] FIG. 5 is a diagram showing the configuration of the first encoding unit according to Embodiment 1. [Figure 6] FIG. 6 is a block diagram of the first encoding unit according to Embodiment 1. [Figure 7] FIG. 7 is a diagram showing the configuration of the first decoding unit according to Embodiment 1. [Figure 8] FIG. 8 is a block diagram of the first decoding unit according to Embodiment 1. [Figure 9] FIG. 9 is a block diagram of the three-dimensional data encoding device according to Embodiment 1. [Figure 10] FIG. 10 is a diagram showing an example of position information according to Embodiment 1. [Figure 11] FIG. 11 is a diagram showing an example of an octree representation of position information according to Embodiment 1. [Figure 12] FIG. 12 is a block diagram of the three-dimensional data decoding device according to Embodiment 1. [Figure 13] FIG. 13 is a block diagram of the attribute information encoding unit according to Embodiment 1. [Figure 14] FIG. 14 is a block diagram of the attribute information decoding unit according to Embodiment 1. [Figure 15] FIG. 15 is a block diagram showing the configuration of the attribute information encoding unit according to Embodiment 1. [Figure 16] FIG. 16 is a block diagram of the attribute information encoding unit according to Embodiment 1. [Figure 17] FIG. 17 is a block diagram showing the configuration of the attribute information decoding unit according to Embodiment 1. [Figure 18] FIG. 18 is a block diagram of the attribute information decoding unit according to Embodiment 1. [Figure 19] FIG. 19 is a diagram showing the configuration of the second encoding unit according to Embodiment 1. [Figure 20]Figure 20 is a block diagram of the second encoding unit according to Embodiment 1. [Figure 21] Figure 21 is a diagram showing the configuration of the second decoding unit according to Embodiment 1. [Figure 22] Figure 22 is a block diagram of the second decoding unit according to Embodiment 1. [Figure 23] Figure 23 is a diagram showing the protocol stack related to PCC encoded data according to Embodiment 1. [Figure 24] Figure 24 shows the configuration of the encoding unit and multiplexing unit according to Embodiment 2. [Figure 25] Figure 25 shows an example of the structure of encoded data according to Embodiment 2. [Figure 26] Figure 26 shows an example of the configuration of encoded data and NAL unit according to Embodiment 2. [Figure 27] Figure 27 shows an example of the semantics of pcc_nal_unit_type according to Embodiment 2. [Figure 28] Figure 28 shows an example of the transmission sequence of the NAL unit according to Embodiment 2. [Figure 29] Figure 29 is a flowchart of the processing performed by the three-dimensional data encoding device according to Embodiment 2. [Figure 30] Figure 30 is a flowchart of the processing performed by the three-dimensional data decoding device according to Embodiment 2. [Figure 31] Figure 31 is a flowchart of the multiplexing process according to Embodiment 2. [Figure 32] Figure 32 is a flowchart of the demultiplexing process according to Embodiment 2. [Figure 33] Figure 33 is a diagram showing the configuration of the distribution system according to Embodiment 3. [Figure 34] Figure 34 shows an example of the configuration of a bitstream of an encoded three-dimensional map according to Embodiment 3. [Figure 35] Figure 35 is a diagram illustrating the effect of improving coding efficiency according to Embodiment 3. [Figure 36]Figure 36 is a flowchart of the server processing according to Embodiment 3. [Figure 37] Figure 37 is a flowchart of the processing performed by the client according to Embodiment 3. [Figure 38] Figure 38 shows an example of the syntax of a submap according to Embodiment 3. [Figure 39] Figure 39 is a schematic diagram showing the encoding type switching process according to Embodiment 3. [Figure 40] Figure 40 shows an example of the syntax of a submap according to Embodiment 3. [Figure 41] Figure 41 is a flowchart of the three-dimensional data encoding process according to Embodiment 3. [Figure 42] Figure 42 is a flowchart of the three-dimensional data decoding process according to Embodiment 3. [Figure 43] Figure 43 is a schematic diagram showing the operation of a modified example of the encoding type switching process according to Embodiment 3. [Figure 44] Figure 44 is a schematic diagram showing the operation of a modified example of the encoding type switching process according to Embodiment 3. [Figure 45] Figure 45 is a schematic diagram showing the operation of a modified example of the encoding type switching process according to Embodiment 3. [Figure 46] Figure 46 is a schematic diagram showing the operation of a modified example of the difference value calculation process according to Embodiment 3. [Figure 47] Figure 47 is a schematic diagram showing the operation of a modified example of the difference value calculation process according to Embodiment 3. [Figure 48] Figure 48 is a schematic diagram showing the operation of a modified example of the difference value calculation process according to Embodiment 3. [Figure 49] Figure 49 is a schematic diagram showing the operation of a modified example of the difference value calculation process according to Embodiment 3. [Figure 50] Figure 50 shows an example of volume syntax according to Embodiment 3. [Figure 51]Figure 51 shows an example of a three-dimensional point according to Embodiment 4. [Figure 52] Figure 52 shows an example of LoD settings according to Embodiment 4. [Figure 53] Figure 53 shows an example of a threshold used to set the LoD according to Embodiment 4. [Figure 54] Figure 54 shows an example of attribute information used for the predicted value according to Embodiment 4. [Figure 55] Figure 55 shows an example of an exponential Golomb code according to Embodiment 4. [Figure 56] Figure 56 is a diagram showing the processing of exponential Golomb codes according to Embodiment 4. [Figure 57] Figure 57 shows an example of the attribute header syntax according to Embodiment 4. [Figure 58] Figure 58 shows an example of attribute data syntax according to Embodiment 4. [Figure 59] Figure 59 is a flowchart of the three-dimensional data encoding process according to Embodiment 4. [Figure 60] Figure 60 is a flowchart of the attribute information encoding process according to Embodiment 4. [Figure 61] Figure 61 is a diagram showing the processing of exponential Golomb codes according to Embodiment 4. [Figure 62] Figure 62 is a diagram showing an example of a reverse lookup table that illustrates the relationship between the remaining reference numerals and their values ​​according to Embodiment 4. [Figure 63] Figure 63 is a flowchart of the three-dimensional data decoding process according to Embodiment 4. [Figure 64] Figure 64 is a flowchart of the attribute information decoding process according to Embodiment 4. [Figure 65] Figure 65 is a block diagram of a three-dimensional data encoding device according to Embodiment 4. [Figure 66] Figure 66 is a block diagram of a three-dimensional data decoding device according to Embodiment 4. [Figure 67]Figure 67 is a diagram showing a first example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 68] Figure 68 shows an example of attribute information used for the predicted value according to Embodiment 5. [Figure 69] Figure 69 is a diagram showing a second example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 70] Figure 70 shows a third example of a table that displays the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 71] Figure 71 is a diagram showing a fourth example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 72] Figure 72 is a diagram showing a fifth example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 73] Figure 73 is a diagram showing a sixth example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 74] Figure 74 is a diagram showing a seventh example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 75] Figure 75 is a diagram showing an eighth example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 76] Figure 76 shows a first example of a binarization table when the prediction mode value according to Embodiment 5 is binarized and encoded. [Figure 77] Figure 77 shows a second example of a binarization table when the predicted mode value according to Embodiment 5 is binarized and encoded. [Figure 78] Figure 78 shows a third example of a binarization table when the predicted mode value according to Embodiment 5 is binarized and encoded. [Figure 79] Figure 79 is a diagram illustrating an example of encoding binary data in a binarization table when the prediction mode according to Embodiment 5 is binarized and encoded. [Figure 80] Figure 80 is a flowchart showing an example of coding the prediction mode value according to Embodiment 5. [Figure 81] Figure 81 is a flowchart showing an example of decoding a prediction mode value according to Embodiment 5. [Figure 82] Figure 82 shows another example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 5. [Figure 83] Figure 83 is a diagram illustrating an example of encoding binary data in a binarization table when the prediction mode according to Embodiment 5 is binarized and encoded. [Figure 84] Figure 84 is a flowchart showing another example of coding the prediction mode value according to Embodiment 5. [Figure 85] Figure 85 is a flowchart showing another example of decoding the predicted mode value according to Embodiment 5. [Figure 86] Figure 86 is a flowchart showing an example of the process for determining whether or not to fix the prediction mode value according to condition A during encoding according to Embodiment 5. [Figure 87] Figure 87 is a flowchart showing an example of the process that determines whether to fix the predicted mode value or decode it according to condition A during decoding according to Embodiment 5. [Figure 88] Figure 88 is a diagram illustrating the maximum absolute difference value maxdiff according to Embodiment 5. [Figure 89] Figure 89 shows an example of syntax according to Embodiment 5. [Figure 90] Figure 90 shows an example of syntax according to Embodiment 5. [Figure 91] Figure 91 is a flowchart of the three-dimensional data encoding process performed by the three-dimensional data encoding device according to Embodiment 5. [Figure 92] Figure 92 is a flowchart of the attribute information encoding process according to Embodiment 5. [Figure 93]Figure 93 is a flowchart of the calculation process for predicted values ​​in the three-dimensional data encoding device according to Embodiment 5. [Figure 94] Figure 94 is a flowchart of the prediction mode selection process according to Embodiment 5. [Figure 95] Figure 95 is a flowchart of the process for selecting the prediction mode that minimizes costs according to Embodiment 5. [Figure 96] Figure 96 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding device according to Embodiment 5. [Figure 97] Figure 97 is a flowchart of the attribute information decoding process according to Embodiment 5. [Figure 98] Figure 98 is a flowchart of the calculation process for predicted values ​​in the three-dimensional data decoding device according to Embodiment 5. [Figure 99] Figure 99 is a flowchart of the decoding process in the prediction mode according to Embodiment 5. [Figure 100] Figure 100 is a block diagram showing the configuration of the attribute information encoding unit of the three-dimensional data encoding device according to Embodiment 5. [Figure 101] Figure 101 is a block diagram showing the configuration of the attribute information decoding unit of the three-dimensional data decoding device according to Embodiment 5. [Figure 102] Figure 102 is a flowchart showing the process of determining the prediction mode performed by the three-dimensional data coding device according to Embodiment 6. [Figure 103] Figure 103 shows an example of the syntax for the process of determining the prediction mode performed by the three-dimensional data encoding device according to Embodiment 6. [Figure 104] Figure 104 is a flowchart showing the process of determining the prediction mode performed by the three-dimensional data decoding device according to Embodiment 6. [Figure 105] Figure 105 shows an example of attribute data syntax when a prediction mode fixed flag according to Embodiment 6 is provided for each three-dimensional point. [Figure 106]Figure 106 shows an example of attribute data syntax when a prediction mode fixing flag according to Embodiment 6 is provided for each level of the LoD. [Figure 107] Figure 107 is a flowchart showing an example of the coding process in the prediction mode of the three-dimensional data coding device according to Embodiment 6. [Figure 108] Figure 108 is a flowchart showing an example of the decoding process in the prediction mode of the three-dimensional data decoding device according to Embodiment 6. [Figure 109] Figure 109 is a flowchart of the three-dimensional data encoding process performed by the three-dimensional data encoding device according to Embodiment 6. [Figure 110] Figure 110 is a flowchart of the attribute information encoding process shown in Figure 109. [Figure 111] Figure 111 is a flowchart of the calculation process for the predicted values ​​shown in Figure 110. [Figure 112] Figure 112 is a flowchart of the selection process for the prediction mode shown in Figure 111. [Figure 113] Figure 113 is a flowchart showing a specific example of the prediction mode selection process shown in Figure 112. [Figure 114] Figure 114 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding device according to Embodiment 6. [Figure 115] Figure 115 is a flowchart of the calculation process for the predicted values ​​shown in Figure 114. [Figure 116] Figure 116 is a flowchart detailing the calculation process for the predicted values ​​shown in Figure 115. [Figure 117] Figure 117 is a flowchart of the calculation process for the prediction mode and quantization value shown in Figure 116. [Figure 118] Figure 118 is a flowchart of the processing when the three-dimensional data encoding device according to Embodiment 6 does not fix the prediction mode. [Figure 119] Figure 119 is a flowchart of the processing when the three-dimensional data decoding device according to Embodiment 6 does not fix the prediction mode. [Figure 120] Figure 120 shows another example of the attribute data syntax according to Embodiment 6. [Figure 121] Figure 121 is a flowchart showing an example of the coding process in the prediction mode of the three-dimensional data coding device according to Embodiment 6. [Figure 122] Figure 122 is a flowchart showing an example of the decoding process in the prediction mode of the three-dimensional data decoding device according to Embodiment 6. [Figure 123] Figure 123 is a flowchart showing another example of the calculation process for predicted values ​​shown in Figure 110. [Figure 124] Figure 124 is a flowchart of the selection process for the prediction mode shown in Figure 123. [Figure 125] Figure 125 is a flowchart showing a specific example of the prediction mode selection process shown in Figure 124. [Figure 126] Figure 126 is a flowchart showing another example of the prediction mode and quantization value calculation process shown in Figure 115. [Figure 127] Figure 127 shows an example of attribute information used for predicting values. [Figure 128] Figure 128 is a diagram showing an example of a reference relationship according to Embodiment 7. [Figure 129] Figure 129 is a diagram showing an example of QW calculation according to Embodiment 7. [Figure 130] Figure 130 shows an example of calculating the predicted residual according to Embodiment 7. [Figure 131] Figure 131 shows an example of a prediction tree used in a three-dimensional data encoding method according to Embodiment 8. [Figure 132] Figure 132 is a flowchart showing an example of a three-dimensional data encoding method according to Embodiment 8. [Figure 133] Figure 133 is a flowchart showing an example of a three-dimensional data decoding method according to Embodiment 8. [Figure 134] Figure 134 is a diagram illustrating the method for generating a prediction tree according to Embodiment 8. [Figure 135]Figure 135 is a diagram illustrating a first example of the prediction mode according to Embodiment 8. [Figure 136] Figure 136 is a diagram showing a second example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 8. [Figure 137] Figure 137 is a diagram showing a second example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 8. [Figure 138] Figure 138 is a diagram showing a third example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 8. [Figure 139] Figure 139 is a diagram showing a fourth example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 8. [Figure 140] Figure 140 is a diagram showing a fifth example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 8. [Figure 141] Figure 141 is a diagram showing a sixth example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 8. [Figure 142] Figure 142 is a diagram showing a seventh example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 8. [Figure 143] Figure 143 shows a first example of a binarization table when the prediction mode value according to Embodiment 8 is binarized and encoded. [Figure 144] Figure 144 shows a second example of a binarization table when the prediction mode value according to Embodiment 8 is binarized and encoded. [Figure 145] Figure 145 shows a third example of a binarization table when the prediction mode value according to Embodiment 8 is binarized and encoded. [Figure 146] Figure 146 is a diagram illustrating an example of encoding binary data in a binarization table when the prediction mode according to Embodiment 8 is binarized and encoded. [Figure 147]Figure 147 is a flowchart showing an example of coding of prediction mode values ​​according to Embodiment 8. [Figure 148] Figure 148 is a flowchart showing an example of decoding a prediction mode value according to Embodiment 8. [Figure 149] Figure 149 is a diagram showing another example of a table that shows the predicted values ​​calculated in each prediction mode according to Embodiment 8. [Figure 150] Figure 150 is a diagram illustrating an example of encoding binary data in a binarization table when the prediction mode according to Embodiment 8 is binarized and encoded. [Figure 151] Figure 151 is a flowchart showing another example of coding the prediction mode value according to Embodiment 8. [Figure 152] Figure 152 is a flowchart showing another example of decoding the predicted mode value according to Embodiment 8. [Figure 153] Figure 153 is a flowchart showing an example of the process for determining whether or not to fix the predicted mode value according to condition A during encoding according to Embodiment 8. [Figure 154] Figure 154 is a flowchart showing an example of the process for determining whether to fix the predicted mode value or decode it according to condition A during decoding according to Embodiment 8. [Figure 155] Figure 155 shows an example of the syntax of the location information header according to Embodiment 8. [Figure 156] Figure 156 shows an example of the syntax of location information according to Embodiment 8. [Figure 157] Figure 157 shows another example of the location information syntax according to Embodiment 8. [Figure 158] Figure 158 shows an example of a prediction tree used in a three-dimensional data encoding method according to Embodiment 9. [Figure 159] Figure 159 shows another example of the location information syntax according to Embodiment 9. [Figure 160]Figure 160 shows an example of the configuration of a prediction tree commonly used for encoding location information and attribute information according to Embodiment 9. [Figure 161] Figure 161 is a flowchart showing an example of a three-dimensional data encoding method according to a modified example of Embodiment 9. [Figure 162] Figure 162 is a flowchart showing an example of a three-dimensional data decoding method according to a modified example of Embodiment 9. [Figure 163] Figure 163 shows an example of the syntax of the attribute information header according to Embodiment 9. [Figure 164] Figure 164 shows another example of the attribute information syntax according to Embodiment 9. [Figure 165] Figure 165 shows an example of the syntax for location information and attribute information according to Embodiment 9. [Figure 166] Figure 166 is a flowchart of the processing of the three-dimensional data encoding device according to Embodiments 8 and 9. [Figure 167] Figure 167 is a flowchart of the processing of the three-dimensional data decoding device according to Embodiments 8 and 9. [Figure 168] Figure 168 is a block diagram of a three-dimensional data creation device according to Embodiment 10. [Figure 169] Figure 169 is a flowchart of the three-dimensional data creation method according to Embodiment 10. [Figure 170] Figure 170 is a diagram showing the configuration of the system according to Embodiment 10. [Figure 171] Figure 171 is a block diagram of the client device according to Embodiment 10. [Figure 172] Figure 172 is a block diagram of the server according to Embodiment 10. [Figure 173] Figure 173 is a flowchart of the three-dimensional data creation process by the client device according to Embodiment 10. [Figure 174] Figure 174 is a flowchart of the sensor information transmission process by the client device according to Embodiment 10. [Figure 175] Figure 175 is a flowchart of the three-dimensional data creation process performed by the server according to Embodiment 10. [Figure 176] Figure 176 is a flowchart of the three-dimensional map transmission process by the server according to Embodiment 10. [Figure 177] Figure 177 shows a modified configuration of the system according to Embodiment 10. [Figure 178] Figure 178 shows the configuration of the server and client device according to Embodiment 10. [Figure 179] Figure 179 is a diagram showing the configuration of the server and client device according to Embodiment 10. [Figure 180] Figure 180 is a flowchart of the processing performed by the client device according to Embodiment 10. [Figure 181] Figure 181 is a diagram showing the configuration of the sensor information collection system according to Embodiment 10. [Figure 182] Figure 182 shows an example of a system according to Embodiment 10. [Figure 183] Figure 183 shows a modified example of the system according to Embodiment 10. [Figure 184] Figure 184 is a flowchart showing an example of application processing according to Embodiment 10. [Figure 185] Figure 185 is a diagram showing the sensor ranges of various sensors according to Embodiment 10. [Figure 186] Figure 186 is a diagram showing an example configuration of an automated driving system according to Embodiment 10. [Figure 187] Figure 187 shows an example of the bitstream configuration according to Embodiment 10. [Figure 188] Figure 188 is a flowchart of the point cloud selection process according to Embodiment 10. [Figure 189] Figure 189 is a diagram showing an example screen of the point cloud selection process according to Embodiment 10. [Figure 190] Figure 190 shows an example of the point cloud selection process screen according to Embodiment 10. [Figure 191] Figure 191 is a diagram showing an example of the point cloud selection process screen according to Embodiment 10. [Modes for carrying out the invention]

[0015] A three-dimensional data encoding method according to one aspect of the present disclosure is a three-dimensional data encoding method for encoding a plurality of three-dimensional points having a hierarchical structure, wherein the method sets one of two or more prediction modes for calculating a predicted value of the first position information of the first three-dimensional point using second position information of one or more second three-dimensional points surrounding a first three-dimensional point, calculates a predicted value of the set prediction mode, calculates a predicted residual which is the difference between the first position information and the calculated predicted value, generates a first bitstream including the set prediction mode and the predicted residual, and in the setting, the prediction mode is set based on the depth of the hierarchical structure of the first three-dimensional point.

[0016] According to this method, location information can be encoded using the predicted value of one prediction mode, which is set based on the depth of the hierarchical structure, out of two or more prediction modes, thus improving the efficiency of encoding location information.

[0017] Furthermore, in the above setting, a prediction mode value may be set that is less than or equal to the depth value of the hierarchical structure of the first three-dimensional point, and the prediction mode value may indicate the prediction mode.

[0018] Furthermore, the first bitstream may also include a prediction mode number indicating the number of the two or more prediction modes.

[0019] Furthermore, in the generation process, the prediction mode value indicating the set prediction mode may be encoded using the number of prediction modes, and the first bitstream may include the encoded prediction mode value as the set prediction mode.

[0020] Furthermore, in the generation process, the predicted mode values ​​may be encoded using a truncated unary code with the number of predicted modes set to its maximum value.

[0021] Therefore, the sign amount of the prediction mode value can be reduced.

[0022] Furthermore, each of the first and second location information includes three elements, and in the above setting, a common prediction mode may be set for the three elements as the one prediction mode for calculating the predicted value of each of the three elements included in the first location information.

[0023] Therefore, the sign amount of the prediction mode value can be reduced.

[0024] Furthermore, each of the first and second location information includes three elements, and in the above setting, an independent prediction mode may be set for each of the three elements as the prediction mode for calculating the predicted value of each of the three elements included in the first location information.

[0025] Therefore, the three-dimensional data decoding device can decode each element independently.

[0026] Furthermore, each of the first and second location information includes three elements, and in the above setting, as the prediction mode for calculating the predicted value of each of the three elements included in the first location information, a prediction mode common to two of the three elements may be set, and a prediction mode independent of the other two elements may be set for the remaining element.

[0027] Therefore, the sign amount of the prediction mode values ​​for two elements can be reduced. In addition, the three-dimensional data decoder can decode the remaining element independently.

[0028] Furthermore, in the generation process, if the number of prediction modes is 1, a second bitstream that does not include the prediction mode value may be generated without encoding the prediction mode value that indicates the prediction mode.

[0029] Therefore, the amount of code in the bitstream can be reduced.

[0030] Furthermore, in the generation process, if a prediction mode is set in which the predicted value calculated in the calculation is 0, a third bitstream that does not include the positive or negative information indicating whether the prediction residual is positive or negative may be generated without encoding the positive or negative information.

[0031] Therefore, the amount of code in the bitstream can be reduced.

[0032] A three-dimensional data decoding method according to one aspect of the present disclosure is a three-dimensional data decoding method for decoding a plurality of three-dimensional points having a hierarchical structure, wherein a first bitstream is obtained which includes an encoded prediction mode of a first three-dimensional point among the plurality of three-dimensional points and an encoded prediction residual, the prediction mode value indicating the encoded prediction mode and the encoded prediction residual are decoded, a prediction value of the prediction mode indicated by the prediction mode value obtained by decoding is calculated, and first position information of the first three-dimensional point is calculated by adding the prediction value and the prediction residual obtained by decoding, wherein the encoded prediction mode included in the first bitstream is a prediction mode set based on the depth of the hierarchical structure of the first three-dimensional point.

[0033] According to this, it is possible to appropriately decode the encoded location information using the predicted value of one prediction mode, which is set based on the depth of the hierarchical structure, from among two or more prediction modes.

[0034] Furthermore, the prediction mode value, which is included in the first bitstream and represents the encoded prediction mode, may be less than or equal to the depth value of the hierarchical structure of the first three-dimensional point.

[0035] Furthermore, the first bitstream may include a prediction mode number indicating the number of prediction modes (two or more).

[0036] Furthermore, in the decoding process, the encoded prediction mode values ​​may be decoded using a truncated unary code with the number of prediction modes set to its maximum value.

[0037] Furthermore, each of the first position information and the second position information of one or more second three-dimensional points surrounding the first three-dimensional point includes three elements, and the prediction mode is used to calculate the predicted value of each of the three elements included in the first position information, and may be set in common for the three elements.

[0038] Furthermore, each of the first position information and the second position information of one or more second three-dimensional points around the first three-dimensional point includes three elements, and the prediction mode is used to calculate the predicted value of each of the three elements included in the first position information, and may be set independently for each of the three elements.

[0039] Furthermore, each of the first position information and the second position information of one or more second three-dimensional points around the first three-dimensional point includes three elements, and the prediction mode is used to calculate the predicted value of each of the three elements included in the first position information, and may be set in common for two of the three elements, while the remaining element is set independently of the other two elements.

[0040] Furthermore, if a second bitstream that does not include the prediction mode value is acquired during the acquisition process, the prediction value may be calculated for a specific prediction mode.

[0041] Furthermore, if a third bitstream is obtained in the acquisition process that does not include positive or negative information indicating whether the predicted residual is positive or negative, the predicted residual may be treated as 0 or a positive number in the calculation of the first position information.

[0042] Furthermore, a three-dimensional data encoding device according to one aspect of the present disclosure is a three-dimensional data encoding device for encoding a plurality of three-dimensional points, comprising a processor and a memory, wherein the processor uses the memory to perform a three-dimensional data encoding method for encoding a plurality of three-dimensional points having a hierarchical structure, which involves setting one of two or more prediction modes for calculating a predicted value of the first position information of the first three-dimensional point using second position information of one or more second three-dimensional points surrounding the first three-dimensional point, calculating the predicted value of the set prediction mode, calculating a predicted residual which is the difference between the first position information and the calculated predicted value, generating a first bitstream including the set prediction mode and the predicted residual, and in the setting, setting the prediction mode based on the depth of the hierarchical structure of the first three-dimensional point.

[0043] According to this method, location information can be encoded using the predicted value of one prediction mode, which is set based on the depth of the hierarchical structure, out of two or more prediction modes, thus improving the efficiency of encoding location information.

[0044] Furthermore, a three-dimensional data decoding device according to one aspect of the present disclosure is a three-dimensional data decoding device for decoding a plurality of three-dimensional points, comprising a processor and a memory, wherein the processor uses the memory to decode a plurality of three-dimensional points having a hierarchical structure, and provides a three-dimensional data decoding method for decoding a plurality of three-dimensional points having a hierarchical structure, which includes acquiring a first bitstream containing an encoded prediction mode of a first three-dimensional point among the plurality of three-dimensional points and an encoded prediction residual, decoding the prediction mode value indicating the encoded prediction mode and the encoded prediction residual, calculating a prediction value of the prediction mode indicated by the prediction mode value obtained by decoding, and calculating first position information of the first three-dimensional point by adding the prediction value and the prediction residual obtained by decoding, wherein the encoded prediction mode included in the first bitstream is a prediction mode set based on the depth of the hierarchical structure of the first three-dimensional point.

[0045] According to this, it is possible to appropriately decode the encoded location information using the predicted value of one prediction mode, which is set based on the depth of the hierarchical structure, from among two or more prediction modes.

[0046] These comprehensive or specific embodiments may be implemented as a system, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, and recording medium.

[0047] The embodiments will be described in detail below with reference to the drawings. Note that the embodiments described below are all specific examples of this disclosure. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept will be described as optional components.

[0048] (Embodiment 1) When using encoded point cloud data in actual devices or services, it is desirable to send and receive necessary information depending on the application in order to reduce network bandwidth. However, until now, such functionality has not existed in the encoded structure of three-dimensional data, nor has there been an encoding method for that purpose.

[0049] This embodiment describes a three-dimensional data encoding method and a three-dimensional data encoding device for providing a function to send and receive information necessary for use in encoded data of a three-dimensional point cloud, a three-dimensional data decoding method and a three-dimensional data decoding device for decoding the encoded data, a three-dimensional data multiplexing method for multiplexing the encoded data, and a three-dimensional data transmission method for transmitting the encoded data.

[0050] In particular, while two encoding methods (encoding schemes) for point cloud data are currently being considered, the structure of the encoded data and the method for storing the encoded data in a system format have not been defined. As a result, there is a problem in that MUX processing (multiplexing), transmission, or storage cannot be performed in the encoding unit.

[0051] Furthermore, there has been no existing method to support formats like PCC (Point Cloud Compression) that use a mixture of two codecs, a first encoding method and a second encoding method.

[0052] This embodiment describes the structure of PCC encoded data in which two codecs, a first encoding method and a second encoding method, coexist, and a method for storing the encoded data in a system format.

[0053] First, the configuration of the three-dimensional data (point cloud data) encoding and decoding system according to this embodiment will be described. Figure 1 is a diagram showing an example of the configuration of the three-dimensional data encoding and decoding system according to this embodiment. As shown in Figure 1, the three-dimensional data encoding and decoding system includes a three-dimensional data encoding system 4601, a three-dimensional data decoding system 4602, a sensor terminal 4603, and an external connection unit 4604.

[0054] The three-dimensional data encoding system 4601 generates encoded data or multiplexed data by encoding point cloud data, which is three-dimensional data. The three-dimensional data encoding system 4601 may be a three-dimensional data encoding device implemented by a single device, or it may be a system implemented by multiple devices. Furthermore, the three-dimensional data encoding device may include some of the multiple processing units included in the three-dimensional data encoding system 4601.

[0055] The three-dimensional data encoding system 4601 includes a point cloud data generation system 4611, a presentation unit 4612, an encoding unit 4613, a multiplexing unit 4614, an input / output unit 4615, and a control unit 4616. The point cloud data generation system 4611 includes a sensor information acquisition unit 4617 and a point cloud data generation unit 4618.

[0056] The sensor information acquisition unit 4617 acquires sensor information from the sensor terminal 4603 and outputs the sensor information to the point cloud data generation unit 4618. The point cloud data generation unit 4618 generates point cloud data from the sensor information and outputs the point cloud data to the encoding unit 4613.

[0057] The display unit 4612 presents sensor information or point cloud data to the user. For example, the display unit 4612 displays information or images based on sensor information or point cloud data.

[0058] The encoding unit 4613 encodes (compresses) the point cloud data and outputs the resulting encoded data, control information obtained during the encoding process, and other additional information to the multiplexing unit 4614. The additional information includes, for example, sensor information.

[0059] The multiplexing unit 4614 generates multiplexed data by multiplexing the encoded data input from the encoding unit 4613, control information, and additional information. The format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.

[0060] The input / output unit 4615 (for example, the communication unit or interface) outputs the multiplexed data to the outside. Alternatively, the multiplexed data is stored in a storage unit such as internal memory. The control unit 4616 (or application execution unit) controls each processing unit. In other words, the control unit 4616 performs control such as encoding and multiplexing.

[0061] The sensor information may also be input to the encoding unit 4613 or the multiplexing unit 4614. Furthermore, the input / output unit 4615 may output the point cloud data or encoded data directly to the outside.

[0062] The transmission signal (multiplexed data) output from the three-dimensional data encoding system 4601 is input to the three-dimensional data decoding system 4602 via the external connection unit 4604.

[0063] The three-dimensional data decoding system 4602 generates point cloud data, which is three-dimensional data, by decoding encoded data or multiplexed data. The three-dimensional data decoding system 4602 may be a three-dimensional data decoding device implemented by a single device, or it may be a system implemented by multiple devices. Furthermore, the three-dimensional data decoding device may include some of the multiple processing units included in the three-dimensional data decoding system 4602.

[0064] The three-dimensional data decoding system 4602 includes a sensor information acquisition unit 4621, an input / output unit 4622, a demultiplexing unit 4623, a decoding unit 4624, a presentation unit 4625, a user interface 4626, and a control unit 4627.

[0065] The sensor information acquisition unit 4621 acquires sensor information from the sensor terminal 4603.

[0066] The input / output unit 4622 acquires the transmission signal, decodes the multiplexed data (file format or packet) from the transmission signal, and outputs the multiplexed data to the demultiplexing unit 4623.

[0067] The demultiplexing unit 4623 acquires encoded data, control information, and additional information from the multiplexed data, and outputs the encoded data, control information, and additional information to the decoding unit 4624.

[0068] The decoding unit 4624 reconstructs the point cloud data by decoding the encoded data.

[0069] The presentation unit 4625 presents point cloud data to the user. For example, the presentation unit 4625 displays information or images based on the point cloud data. The user interface 4626 acquires instructions based on user operations. The control unit 4627 (or application execution unit) controls each processing unit. In other words, the control unit 4627 performs control such as demultiplexing, decoding, and presentation.

[0070] The input / output unit 4622 may acquire point cloud data or encoded data directly from an external source. The presentation unit 4625 may acquire additional information such as sensor information and present information based on that additional information. The presentation unit 4625 may also make presentations based on user instructions acquired through the user interface 4626.

[0071] The sensor terminal 4603 generates sensor information, which is information obtained from the sensor. The sensor terminal 4603 is a terminal equipped with a sensor or camera, and may be, for example, a mobile object such as an automobile, an aerial object such as an airplane, a mobile terminal, or a camera.

[0072] The sensor information that can be acquired by the sensor terminal 4603 includes, for example, (1) the distance between the sensor terminal 4603 and the object, or the reflectivity of the object, obtained from a LiDAR, millimeter-wave radar, or infrared sensor, and (2) the distance between the camera and the object, or the reflectivity of the object, obtained from multiple monocular camera images or stereo camera images. The sensor information may also include the sensor's attitude, orientation, gyroscope (angular velocity), position (GPS information or altitude), speed, or acceleration. The sensor information may also include temperature, atmospheric pressure, humidity, or magnetism.

[0073] The external connection unit 4604 is implemented by an integrated circuit (LSI or IC), an external storage unit, communication with a cloud server via the internet, or broadcasting, etc.

[0074] Next, we will explain point cloud data. Figure 2 shows the structure of point cloud data. Figure 3 shows an example of the structure of a data file containing information about point cloud data.

[0075] Point cloud data contains data for multiple points. Each point's data includes location information (three-dimensional coordinates) and attribute information related to that location. A collection of these points is called a point cloud. For example, a point cloud represents the three-dimensional shape of an object.

[0076] Position information, such as three-dimensional coordinates, is sometimes referred to as geometry. Furthermore, the data for each point may include attribute information of multiple attribute types. Attribute types include, for example, color or reflectance.

[0077] One location information may be associated with one attribute information, or multiple attribute information of different attribute types may be associated with one location information. Furthermore, multiple attribute information of the same attribute type may be associated with one location information.

[0078] The example data file structure shown in Figure 3 represents a case where location information and attribute information correspond one-to-one, and it shows the location information and attribute information of the N points that make up the point cloud data.

[0079] Location information includes, for example, information for the three axes: x, y, and z. Attribute information includes, for example, RGB color information. A typical data file is a ply file.

[0080] Next, we will explain the types of point cloud data. Figure 4 is a diagram illustrating the types of point cloud data. As shown in Figure 4, point cloud data includes static objects and dynamic objects.

[0081] A static object is three-dimensional point cloud data for any given time (a specific moment). A dynamic object is three-dimensional point cloud data that changes over time. Hereafter, three-dimensional point cloud data for a given time will be referred to as a PCC frame, or simply a frame.

[0082] The object can be a point cloud with a somewhat limited area, like regular video data, or it can be a large-scale point cloud with no area limitations, like map information.

[0083] Furthermore, point cloud data of various densities may exist, including both sparse and dense point cloud data.

[0084] The details of each processing unit are described below. Sensor information is acquired by various methods, such as distance sensors like LIDAR or rangefinders, stereo cameras, or combinations of multiple monocular cameras. The point cloud data generation unit 4618 generates point cloud data based on the sensor information obtained by the sensor information acquisition unit 4617. The point cloud data generation unit 4618 generates position information as point cloud data and adds attribute information to the position information.

[0085] The point cloud data generation unit 4618 may process the point cloud data when generating position information or adding attribute information. For example, the point cloud data generation unit 4618 may reduce the amount of data by deleting point clouds with overlapping positions. The point cloud data generation unit 4618 may also transform the position information (such as position shifting, rotation, or normalization) or render the attribute information.

[0086] In Figure 1, the point cloud data generation system 4611 is included in the three-dimensional data encoding system 4601, but it may also be provided independently outside of the three-dimensional data encoding system 4601.

[0087] The encoding unit 4613 generates encoded data by encoding the point cloud data based on a predetermined encoding method. There are two main types of encoding methods. The first is an encoding method using positional information, which will be referred to as the first encoding method hereafter. The second is an encoding method using a video codec, which will be referred to as the second encoding method hereafter.

[0088] The decoding unit 4624 decodes the point cloud data by decoding the encoded data based on a predetermined encoding method.

[0089] The multiplexing unit 4614 generates multiplexed data by multiplexing the encoded data using an existing multiplexing method. The generated multiplexed data is transmitted or stored. In addition to PCC encoded data, the multiplexing unit 4614 multiplexes other media such as video, audio, subtitles, applications, files, or reference time information. Furthermore, the multiplexing unit 4614 may also multiplex attribute information related to sensor information or point cloud data.

[0090] Multiplexing methods or file formats include ISOBMFF, ISOBMFF-based transmission methods such as MPEG-DASH, MMT, MPEG-2 TS Systems, and RMP.

[0091] The demultiplexing unit 4623 extracts PCC encoded data, other media, and time information from the multiplexed data.

[0092] The input / output unit 4615 transmits the multiplexed data using a method appropriate to the transmission medium or storage medium, such as broadcasting or communication. The input / output unit 4615 may communicate with other devices via the internet or with storage units such as cloud servers.

[0093] Communication protocols such as HTTP, FTP, TCP, or UDP can be used. Either a pull-type or push-type communication method may be employed.

[0094] Either wired or wireless transmission may be used. Wired transmission methods include Ethernet®, USB, RS-232C, HDMI®, or coaxial cable. Wireless transmission methods include wireless LAN, Wi-Fi®, Bluetooth®, or millimeter wave.

[0095] Furthermore, broadcasting formats such as DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 may be used.

[0096] Figure 5 shows the configuration of a first encoding unit 4630, which is an example of an encoding unit 4613 that performs encoding using the first encoding method. Figure 6 is a block diagram of the first encoding unit 4630. The first encoding unit 4630 generates encoded data (encoded stream) by encoding point cloud data using the first encoding method. This first encoding unit 4630 includes a location information encoding unit 4631, an attribute information encoding unit 4632, an additional information encoding unit 4633, and a multiplexing unit 4634.

[0097] The first encoding unit 4630 is characterized by performing encoding while being aware of the three-dimensional structure. Furthermore, the first encoding unit 4630 is characterized by the attribute information encoding unit 4632 performing encoding using information obtained from the location information encoding unit 4631. The first encoding method is also called GPCC (Geometry-based PCC).

[0098] The point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and includes position information, attribute information, and other additional information (metadata). The position information is input to the position information encoding unit 4631, the attribute information is input to the attribute information encoding unit 4632, and the additional information is input to the additional information encoding unit 4633.

[0099] The location information encoding unit 4631 generates encoded location information (Compressed Geometry), which is encoded data, by encoding location information. For example, the location information encoding unit 4631 encodes location information using an N-tree structure such as an octree. Specifically, in an octree, the target space is divided into 8 nodes (subspaces), and 8 bits of information (occupancy code) are generated to indicate whether or not a point cloud is contained in each node. Furthermore, nodes containing point clouds are further divided into 8 nodes, and 8 bits of information are generated to indicate whether or not a point cloud is contained in each of these 8 nodes. This process is repeated until the number of point clouds contained in a predetermined hierarchy or node falls below a threshold.

[0100] The attribute information encoding unit 4632 generates encoded attribute information (Compressed Attribute), which is encoded data, by encoding it using the configuration information generated by the location information encoding unit 4631. For example, the attribute information encoding unit 4632 determines the reference point (reference node) to be referenced in encoding the target point (target node) to be processed, based on the octave tree structure generated by the location information encoding unit 4631. For example, the attribute information encoding unit 4632 references a surrounding node or adjacent node whose parent node in the octave tree is the same as the target node. Note that the method for determining the reference relationship is not limited to this.

[0101] Furthermore, the attribute information encoding process may include at least one of the following: quantization, prediction, and arithmetic encoding. In this case, a reference means using a reference node to calculate the predicted value of the attribute information, or using the state of a reference node (for example, occupancy information indicating whether or not the reference node contains a point cloud) to determine the encoding parameters. For example, encoding parameters may be quantization parameters in the quantization process, or context in arithmetic encoding.

[0102] The additional information encoding unit 4633 generates encoded data, or compressed additional information (Compressed MetaData), by encoding the compressible data from the additional information.

[0103] The multiplexing unit 4634 generates a compressed stream, which is encoded data, by multiplexing encoded position information, encoded attribute information, encoded additional information, and other additional information. The generated compressed stream is output to a processing unit of the system layer (not shown).

[0104] Next, we will describe a first decoding unit 4640, which is an example of a decoding unit 4624 that performs decoding of the first encoding method. Figure 7 is a diagram showing the configuration of the first decoding unit 4640. Figure 8 is a block diagram of the first decoding unit 4640. The first decoding unit 4640 generates point cloud data by decoding the encoded data (encoded stream) encoded by the first encoding method using the first encoding method. This first decoding unit 4640 includes a demultiplexing unit 4641, a location information decoding unit 4642, an attribute information decoding unit 4643, and an additional information decoding unit 4644.

[0105] A compressed stream, which is encoded data, is input to the first decoding unit 4640 from a processing unit of the system layer (not shown).

[0106] The demultiplexing unit 4641 separates encoded location information (Compressed Geometry), encoded attribute information (Compressed Attribute), encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0107] The location information decoding unit 4642 generates location information by decoding the encoded location information. For example, the location information decoding unit 4642 reconstructs the location information of a point cloud represented by three-dimensional coordinates from encoded location information represented by an N-tree structure such as an octree.

[0108] The attribute information decoding unit 4643 decodes the encoded attribute information based on the configuration information generated by the location information decoding unit 4642. For example, the attribute information decoding unit 4643 determines the reference point (reference node) to be referenced in the decoding of the target point (target node) to be processed, based on the octave tree structure obtained by the location information decoding unit 4642. For example, the attribute information decoding unit 4643 references a surrounding node or adjacent node whose parent node in the octave tree is the same as the target node. Note that the method for determining the reference relationship is not limited to this.

[0109] Furthermore, the attribute information decoding process may include at least one of the following: inverse quantization, prediction, and arithmetic decoding. In this case, "reference" means using a reference node to calculate the predicted value of the attribute information, or using the state of the reference node (for example, occupancy information indicating whether or not the reference node contains a point cloud) to determine the decoding parameters. For example, decoding parameters may be quantization parameters in the inverse quantization process, or context in arithmetic decoding.

[0110] The additional information decoding unit 4644 generates additional information by decoding the encoded additional information. The first decoding unit 4640 uses the additional information necessary for decoding location information and attribute information during decoding and outputs the additional information necessary for the application to the outside.

[0111] Next, an example of the configuration of the location information coding unit will be described. Figure 9 is a block diagram of the location information coding unit 2700 according to this embodiment. The location information coding unit 2700 comprises an octree generation unit 2701, a geometric information calculation unit 2702, a coding table selection unit 2703, and an entropy coding unit 2704.

[0112] The octree generation unit 2701 generates an octree from the input position information and generates occupancy codes for each node in the octree. The geometric information calculation unit 2702 obtains information indicating whether the adjacent nodes of the target node are occupied nodes or not. For example, the geometric information calculation unit 2702 calculates the occupancy information of adjacent nodes (information indicating whether the adjacent node is an occupied node or not) from the occupancy code of the parent node to which the target node belongs. Alternatively, the geometric information calculation unit 2702 may store the encoded nodes in a list and search for adjacent nodes from that list. The geometric information calculation unit 2702 may also switch adjacent nodes depending on the position of the target node within the parent node.

[0113] The coding table selection unit 2703 selects a coding table to be used for entropy coding of the target node using the occupancy information of adjacent nodes calculated by the geometric information calculation unit 2702. For example, the coding table selection unit 2703 may generate a bit sequence using the occupancy information of adjacent nodes and select a coding table with an index number generated from that bit sequence.

[0114] The entropy coding unit 2704 generates coded location information and metadata by performing entropy coding on the occupancy code of the target node using the coding table of the selected index number. The entropy coding unit 2704 may also add information indicating the selected coding table to the coded location information.

[0115] The following describes the octree representation and the scanning order of location information. Location information (location data) is converted into an octree structure (octreeization) and then encoded. An octree structure consists of nodes and leaves. Each node has eight nodes or leaves, and each leaf has voxel (VXL) information. Figure 10 shows an example of the structure of location information containing multiple voxels. Figure 11 shows an example of the location information shown in Figure 10 converted into an octree structure. Here, among the leaves shown in Figure 11, leaves 1, 2, and 3 represent the voxels VXL1, VXL2, and VXL3 shown in Figure 10, respectively, and represent VXL containing the point cloud (hereinafter referred to as effective VXL).

[0116] Specifically, node 1 corresponds to the overall space encompassing the positional information in Figure 10. The overall space corresponding to node 1 is divided into eight nodes, and of these eight nodes, the node containing the valid VXL is further divided into eight nodes or leaves, and this process is repeated for each level of the tree structure. Here, each node corresponds to a subspace and holds information (occupancy code) indicating the position of the next node or leaf after the division as node information. In addition, the lowest-level block is set as a leaf, and leaf information such as the number of points contained within the leaf is held.

[0117] Next, an example of the configuration of the location information decoding unit will be described. Figure 12 is a block diagram of the location information decoding unit 2710 according to this embodiment. The location information decoding unit 2710 comprises an octree generation unit 2711, a geometric information calculation unit 2712, an encoding table selection unit 2713, and an entropy decoding unit 2714.

[0118] The octane tree generation unit 2711 generates an octane tree of a given space (node) using the header information or metadata of the bitstream. For example, the octane tree generation unit 2711 generates a large space (root node) using the size of the x, y, and z axes of a given space attached to the header information, and then generates an octane tree by dividing that space into two in each of the x, y, and z axes to generate eight small spaces A (nodes A0 to A7). Also, nodes A0 to A7 are set in order as the target nodes.

[0119] The geometric information calculation unit 2712 obtains occupancy information indicating whether an adjacent node to the target node is an occupied node. For example, the geometric information calculation unit 2712 calculates the occupancy information of an adjacent node from the occupancy code of the parent node to which the target node belongs. Alternatively, the geometric information calculation unit 2712 may store the decoded nodes in a list and search for adjacent nodes from that list. The geometric information calculation unit 2712 may also switch adjacent nodes depending on the position of the target node within its parent node.

[0120] The coding table selection unit 2713 selects a coding table (decoding table) to be used for entropy decoding of the target node using the occupancy information of adjacent nodes calculated by the geometric information calculation unit 2712. For example, the coding table selection unit 2713 may generate a bit sequence using the occupancy information of adjacent nodes and select a coding table with an index number generated from that bit sequence.

[0121] The entropy decoding unit 2714 generates location information by entropy decoding the occupancy code of the target node using the selected coding table. Alternatively, the entropy decoding unit 2714 may decode and obtain the information of the selected coding table from the bitstream, and then entropy decode the occupancy code of the target node using the coding table indicated by that information.

[0122] The configuration of the attribute information encoding unit and the attribute information decoding unit will be described below. Figure 13 is a block diagram showing an example configuration of the attribute information encoding unit A100. The attribute information encoding unit may include multiple encoding units that perform different encoding methods. For example, the attribute information encoding unit may switch between the following two methods depending on the use case.

[0123] The attribute information encoding unit A100 includes the LoD attribute information encoding unit A101 and the transformation attribute information encoding unit A102. The LoD attribute information encoding unit A101 uses the positional information of the three-dimensional points to classify each three-dimensional point into multiple layers, predicts the attribute information of the three-dimensional points belonging to each layer, and encodes the predicted residual. Here, each classified layer is called LoD (Level of Detail).

[0124] The attribute information encoding unit A102 encodes attribute information using RAHT (Region Adaptive Hierarchical Transform). Specifically, the attribute information encoding unit A102 applies RAHT or Haar transform to each attribute information based on the position information of three-dimensional points to generate high-frequency and low-frequency components of each layer, and encodes these values ​​using quantization and entropy coding, etc.

[0125] Figure 14 is a block diagram showing an example configuration of the attribute information decoding unit A110. The attribute information decoding unit may include multiple decoding units that perform different decoding methods. For example, the attribute information decoding unit may decode by switching between the following two methods based on the information contained in the header and metadata.

[0126] The attribute information decoding unit A110 includes the LoD attribute information decoding unit A111 and the converted attribute information decoding unit A112. The LoD attribute information decoding unit A111 classifies each three-dimensional point into multiple layers using the positional information of the three-dimensional points, and decodes the attribute values ​​while predicting the attribute information of the three-dimensional points belonging to each layer.

[0127] The attribute information decoding unit A112 decodes attribute information using RAHT (Region Adaptive Hierarchical Transform). Specifically, the attribute information decoding unit A112 decodes attribute values ​​by applying inverse RAHT or inverse Haar transform to the high-frequency and low-frequency components of each attribute value based on the position information of the three-dimensional points.

[0128] Figure 15 is a block diagram showing the configuration of an attribute information encoding unit 3140, which is an example of an LoD attribute information encoding unit A101.

[0129] The attribute information coding unit 3140 includes an LoD generation unit 3141, a surrounding search unit 3142, a prediction unit 3143, a prediction residual calculation unit 3144, a quantization unit 3145, an arithmetic coding unit 3146, an inverse quantization unit 3147, a decoded value generation unit 3148, and a memory 3149.

[0130] The LoD generation unit 3141 generates LoDs using the positional information of three-dimensional points.

[0131] The surrounding search unit 3142 uses the LoD generation result from the LoD generation unit 3141 and distance information indicating the distance between each three-dimensional point to search for neighboring three-dimensional points adjacent to each three-dimensional point.

[0132] The prediction unit 3143 generates predicted values ​​for the attribute information of the target three-dimensional point to be encoded.

[0133] The prediction residual calculation unit 3144 calculates (generates) the prediction residual of the predicted value of the attribute information generated by the prediction unit 3143.

[0134] The quantization unit 3145 quantizes the predicted residuals of the attribute information calculated by the predicted residual calculation unit 3144.

[0135] The arithmetic coding unit 3146 arithmetically codes the predicted residuals after quantization by the quantization unit 3145. The arithmetic coding unit 3146 outputs the bitstream containing the arithmetic coded predicted residuals to, for example, a three-dimensional data decoding device.

[0136] The predicted residual may be binarized, for example, by the quantization unit 3145 before being arithmetic-coded by the arithmetic coding unit 3146.

[0137] Furthermore, for example, the arithmetic coding unit 3146 may initialize the coding table used for arithmetic coding before arithmetic coding. The arithmetic coding unit 3146 may initialize the coding table used for arithmetic coding for each layer. In addition, the arithmetic coding unit 3146 may output information indicating the position of the layer in which the coding table was initialized in the bitstream.

[0138] The inverse quantization unit 3147 inversely quantizes the predicted residual after it has been quantized by the quantization unit 3145.

[0139] The decoded value generation unit 3148 generates a decoded value by adding the predicted value of the attribute information generated by the prediction unit 3143 and the predicted residual after inverse quantization by the inverse quantization unit 3147.

[0140] Memory 3149 is a memory that stores the decoded values ​​of the attribute information of each three-dimensional point decoded by the decoded value generation unit 3148. For example, when the prediction unit 3143 generates prediction values ​​for three-dimensional points that have not yet been encoded, it uses the decoded values ​​of the attribute information of each three-dimensional point stored in memory 3149 to generate the prediction values.

[0141] Figure 16 is a block diagram of an attribute information encoding unit 6600, which is an example of a conversion attribute information encoding unit A102. The attribute information encoding unit 6600 comprises a sorting unit 6601, a Haar conversion unit 6602, a quantization unit 6603, an inverse quantization unit 6604, an inverse Haar conversion unit 6605, a memory 6606, and an arithmetic encoding unit 6607.

[0142] The sorting unit 6601 generates Morton codes using the position information of three-dimensional points and sorts multiple three-dimensional points in Morton code order. The Haar transform unit 6602 generates coding coefficients by applying the Haar transform to the attribute information. The quantization unit 6603 quantizes the coding coefficients of the attribute information.

[0143] The inverse quantization unit 6604 inversely quantizes the encoded coefficients after quantization. The inverse Haar transform unit 6605 applies the inverse Haar transform to the encoded coefficients. The memory 6606 stores the attribute information values ​​of multiple decoded three-dimensional points. For example, the attribute information of decoded three-dimensional points stored in the memory 6606 may be used for predicting unencoded three-dimensional points.

[0144] The arithmetic coding unit 6607 calculates ZeroCnt from the quantized coding coefficients and arithmetically codes ZeroCnt. The arithmetic coding unit 6607 also arithmetically codes the non-zero coding coefficients after quantization. The arithmetic coding unit 6607 may also binarize the coding coefficients before arithmetic coding. Furthermore, the arithmetic coding unit 6607 may generate and code various header information.

[0145] Figure 17 is a block diagram showing the configuration of an attribute information decoding unit 3150, which is an example of an LoD attribute information decoding unit A111.

[0146] The attribute information decoding unit 3150 includes an LoD generation unit 3151, a surrounding search unit 3152, a prediction unit 3153, an arithmetic decoding unit 3154, an inverse quantization unit 3155, a decoded value generation unit 3156, and a memory 3157.

[0147] The LoD generation unit 3151 generates LoDs using the position information of three-dimensional points decoded by the position information decoding unit (not shown in Figure 17).

[0148] The surrounding search unit 3152 uses the LoD generation result from the LoD generation unit 3151 and distance information indicating the distance between each three-dimensional point to search for neighboring three-dimensional points adjacent to each three-dimensional point.

[0149] The prediction unit 3153 generates predicted values ​​for the attribute information of the target three-dimensional point to be decoded.

[0150] The arithmetic decoding unit 3154 arithmetically decodes the predicted residuals in the bitstream obtained from the attribute information encoding unit 3140 shown in Figure 15. The arithmetic decoding unit 3154 may initialize the decoding table used for arithmetic decoding. The arithmetic decoding unit 3154 initializes the decoding table used for arithmetic decoding for the layers encoded by the arithmetic encoding unit 3146 shown in Figure 15. The arithmetic decoding unit 3154 may initialize the decoding table for each layer. Alternatively, the arithmetic decoding unit 3154 may initialize the decoding table based on information in the bitstream indicating the position of the layer whose encoding table has been initialized.

[0151] The inverse quantization unit 3155 inversely quantizes the predicted residuals that were arithmetically decoded by the arithmetic decoding unit 3154.

[0152] The decoded value generation unit 3156 generates a decoded value by adding the predicted value generated by the prediction unit 3153 and the predicted residual after inverse quantization by the inverse quantization unit 3155. The decoded value generation unit 3156 outputs the decoded attribute information data to another device.

[0153] Memory 3157 is a memory that stores the decoded values of the attribute information of each three-dimensional point decoded by the decoded value generation unit 3156. For example, when the prediction unit 3153 generates a predicted value of a three-dimensional point that has not yet been decoded, it uses the decoded values of the attribute information of each three-dimensional point stored in the memory 3157 to generate a predicted value.

[0154] FIG. 18 is a block diagram of an attribute information decoding unit 6610 which is an example of the conversion attribute information decoding unit A112. The attribute information decoding unit 6610 includes an arithmetic decoding unit 6611, an inverse quantization unit 6612, an inverse Haar transform unit 6613, and a memory 6614.

[0155] The arithmetic decoding unit 6611 arithmetically decodes the ZeroCnt and the encoding coefficients included in the bit stream. Note that the arithmetic decoding unit 6611 may decode various header information.

[0156] The inverse quantization unit 6612 inverse quantizes the arithmetically decoded encoding coefficients. The inverse Haar transform unit 6613 applies an inverse Haar transform to the encoding coefficients after inverse quantization. The memory 6614 stores the values of the attribute information of a plurality of decoded three-dimensional points. For example, the decoded attribute information of the three-dimensional points stored in the memory 6614 may be used for predicting three-dimensional points that have not been decoded.

[0157] Next, a second encoding unit 4650 which is an example of an encoding unit 4613 that performs encoding using the second encoding method will be described. FIG. 19 is a diagram showing the configuration of the second encoding unit 4650. FIG. 20 is a block diagram of the second encoding unit 4650. [[ID=I8]]

[0158] The second encoding unit 4650 generates encoded data (encoded stream) by encoding the point cloud data using the second encoding method. This second encoding unit 4650 includes an additional information generation unit 4651, a position image generation unit 4652, an attribute image generation unit4653, a video encoding unit 4654, an additional information encoding unit 4655, and a multiplexing unit 4656.

[0159] The second encoding unit 4650 generates a position image and an attribute image by projecting a three-dimensional structure onto a two-dimensional image, and encodes the generated position image and attribute image using an existing video encoding method. The second encoding method is also called VPCC (Video based PCC).

[0160] The point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData).

[0161] The additional information generation unit 4651 generates map information of a plurality of two-dimensional images by projecting a three-dimensional structure onto a two-dimensional image.

[0162] The position image generation unit 4652 generates a position image (Geometry Image) based on the position information and the map information generated by the additional information generation unit 4651. This position image is, for example, a distance image in which distance (Depth) is shown as a pixel value. Note that this distance image may be an image of a plurality of point clouds viewed from one viewpoint (an image obtained by projecting a plurality of point clouds onto one two-dimensional plane), or a plurality of images of a plurality of point clouds viewed from a plurality of viewpoints, or one image obtained by integrating these plurality of images.

[0163] The attribute image generation unit 4653 generates an attribute image based on the attribute information and the map information generated by the additional information generation unit 4651. This attribute image is, for example, an image in which attribute information (e.g., color (RGB)) is shown as a pixel value. Note that this image may be an image of a plurality of point clouds viewed from one viewpoint (an image obtained by projecting a plurality of point clouds onto one two-dimensional plane), or a plurality of images of a plurality of point clouds viewed from a plurality of viewpoints, or one image obtained by integrating these plurality of images.

[0164] The video encoding unit 4654 generates encoded data, namely a compressed geometric image and a compressed attribute image, by encoding the position image and attribute image using a video encoding scheme. Any known encoding scheme may be used as the video encoding scheme. For example, the video encoding scheme may be AVC or HEVC.

[0165] The additional information encoding unit 4655 generates encoded additional information (Compressed MetaData) by encoding additional information and map information included in the point cloud data.

[0166] The multiplexing unit 4656 generates a compressed stream, which is encoded data, by multiplexing the encoded position image, encoded attribute image, encoded additional information, and other additional information. The generated compressed stream is output to a processing unit of the system layer (not shown).

[0167] Next, we will describe a second decoding unit 4660, which is an example of a decoding unit 4624 that performs decoding of the second encoding method. Figure 21 is a diagram showing the configuration of the second decoding unit 4660. Figure 22 is a block diagram of the second decoding unit 4660. The second decoding unit 4660 generates point cloud data by decoding the encoded data (encoded stream) encoded by the second encoding method using the second encoding method. This second decoding unit 4660 includes a demultiplexing unit 4661, a video decoding unit 4662, an additional information decoding unit 4663, a location information generation unit 4664, and an attribute information generation unit 4665.

[0168] A compressed stream, which is encoded data, is input to the second decoding unit 4660 from a processing unit of the system layer (not shown).

[0169] The demultiplexing unit 4661 separates the encoded location image (Compressed Geometry Image), encoded attribute image (Compressed Attribute Image), encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0170] The video decoding unit 4662 generates a position image and an attribute image by decoding the encoded position image and the encoded attribute image using a video encoding scheme. Any known encoding scheme may be used as the video encoding scheme. For example, the video encoding scheme may be AVC or HEVC.

[0171] The additional information decoding unit 4663 generates additional information, including map information, by decoding the encoded additional information.

[0172] The location information generation unit 4664 generates location information using the location image and map information. The attribute information generation unit 4665 generates attribute information using the attribute image and map information.

[0173] The second decoding unit 4660 uses the additional information necessary for decoding during the decoding process and outputs the additional information necessary for the application to the outside.

[0174] The following describes the challenges in the PCC encoding scheme. Figure 23 is a diagram showing the protocol stack involved in PCC encoded data. Figure 23 shows an example in which data from other media, such as video (e.g., HEVC) or audio, is multiplexed onto PCC encoded data and then transmitted or stored.

[0175] Multiplexing schemes and file formats have the function of multiplexing, transmitting, or storing various encoded data. In order to transmit or store encoded data, the encoded data must be converted into the format of the multiplexing scheme. For example, HEVC specifies a technique in which encoded data is stored in a data structure called a NAL unit, and the NAL unit is stored in ISOBMFF.

[0176] On the one hand, currently, as encoding methods for point cloud data, a first encoding method (Codec1) and a second encoding method (Codec2) are being considered. However, the configuration of the encoded data and the method of storing the encoded data in the system format are not defined. As it is, there is a problem that MUX processing (multiplexing), transmission, and storage in the encoding unit cannot be performed.

[0177] In the following, unless otherwise specified, any description of a specific encoding method shall indicate either the first encoding method or the second encoding method.

[0178] (Embodiment 2) In this embodiment, the types of encoded data (Geometry, Attribute, Metadata) generated by the above-described first encoding unit 4630 or second encoding unit 4650, the method of generating additional information (metadata), and the multiplexing process in the multiplexing unit will be described. Note that the additional information (metadata) may also be referred to as a parameter set or control information.

[0179] In this embodiment, the dynamic object (three-dimensional point cloud data that changes over time) described in FIG. 4 will be used as an example for explanation. However, the same method may also be used for a static object (three-dimensional point cloud data at any time).

[0180] FIG. 24 is a diagram showing the configuration of an encoding unit 4801 and a multiplexing unit 4802 included in the three-dimensional data encoding apparatus according to this embodiment. The encoding unit 4801 corresponds to, for example, the above-described first encoding unit 4630 or second encoding unit 4650. The multiplexing unit 4802 corresponds to the above-described multiplexing unit 4634 or 4656.

[0181] The encoding unit 4801 encodes the point cloud data of a plurality of PCC (Point Cloud Compression) frames and generates encoded data (Multiple Compressed Data) of a plurality of position information, attribute information, and additional information.

[0182] The multiplexing unit 4802 converts data of multiple data types (location information, attribute information, and additional information) into NAL units, thereby transforming the data into a data configuration that takes into account data access by the decoding device.

[0183] Figure 25 shows an example of the structure of encoded data generated by the encoding unit 4801. The arrows in the figure indicate dependencies related to the decoding of encoded data, with the source of the arrow depending on the data at the end of the arrow. In other words, the decoding device decodes the data at the end of the arrow and uses that decoded data to decode the source of the arrow. To put it another way, dependency means that the dependent data is referenced (used) in the processing of the dependent data (encoding or decoding, etc.).

[0184] First, let's explain the process of generating encoded location data. The encoding unit 4801 generates compressed location data (Compressed Geometry Data) for each frame by encoding the location information of each frame. The encoded location data is represented by G(i), where i represents the frame number or the time of the frame.

[0185] Furthermore, the encoding unit 4801 generates a position parameter set (GPS(i)) corresponding to each frame. The position parameter set includes parameters that can be used to decode the encoded position data. Also, the encoded position data for each frame depends on the corresponding position parameter set.

[0186] Furthermore, encoded position data consisting of multiple frames is defined as a position sequence (Geometry Sequence). The encoding unit 4801 generates a position sequence parameter set (Geometry Sequence PS: also written as Position SPS) that stores parameters commonly used for decoding multiple frames within the position sequence. The position sequence depends on the Position SPS.

[0187] Next, the process for generating encoded attribute data will be explained. The encoding unit 4801 generates compressed attribute data for each frame by encoding the attribute information of each frame. The compressed attribute data is represented by A(i). Figure 25 shows an example where attribute X and attribute Y exist, with the compressed attribute data for attribute X represented by AX(i) and the compressed attribute data for attribute Y represented by AY(i).

[0188] Furthermore, the encoding unit 4801 generates an attribute parameter set (APS(i)) corresponding to each frame. The attribute parameter set for attribute X is represented by AXPS(i), and the attribute parameter set for attribute Y is represented by AYPS(i). The attribute parameter set includes parameters that can be used to decode the encoded attribute information. The encoded attribute data depends on the corresponding attribute parameter set.

[0189] Furthermore, encoded attribute data consisting of multiple frames is defined as an attribute sequence. The encoding unit 4801 generates an attribute sequence parameter set (Attribute Sequence PS, also written as attribute SPS) that stores parameters commonly used for decoding multiple frames within the attribute sequence. The attribute sequence depends on the attribute SPS.

[0190] Furthermore, in the first encoding method, the encoded attribute data depends on the encoded position data.

[0191] Figure 25 also shows an example where there are two types of attribute information (attribute X and attribute Y). When there are two types of attribute information, for example, two encoding units generate the respective data and metadata. Also, for example, an attribute sequence is defined for each type of attribute information, and an attribute SPS is generated for each type of attribute information.

[0192] Note that Figure 25 shows an example where there is one type of positional information and two types of attribute information, but the example is not limited to this; there may be one type of attribute information or three or more types. In this case as well, encoded data can be generated in the same way. Furthermore, in the case of point cloud data that does not have attribute information, attribute information is not required. In that case, the encoding unit 4801 does not need to generate a parameter set related to attribute information.

[0193] Next, the process of generating additional information (metadata) will be described. The encoding unit 4801 generates a PCC stream PS (also written as Stream PS), which is a parameter set for the entire PCC stream. The encoding unit 4801 stores in Stream PS parameters that can be used in common for decoding one or more location sequences and one or more attribute sequences. For example, Stream PS includes identification information indicating the codec of the point cloud data, and information indicating the algorithm used for encoding. The location sequences and attribute sequences depend on Stream PS.

[0194] Next, we will explain the Access Unit and GOF. In this embodiment, we introduce the new concepts of Access Unit (AU) and GOF (Group of Frame).

[0195] An access unit is the basic unit for accessing data during decryption, and consists of one or more data points and one or more metadata points. For example, an access unit consists of location information at the same time and one or more attribute information points. A GOF (Group of Four) is a random access unit and consists of one or more access units.

[0196] The encoding unit 4801 generates an access unit header (AU Header) as identification information indicating the beginning of an access unit. The encoding unit 4801 stores parameters related to the access unit in the access unit header. For example, the access unit header includes the structure or information of the encoded data contained in the access unit. The access unit header also includes parameters commonly used in the data contained in the access unit, such as parameters related to the decoding of the encoded data.

[0197] The encoding unit 4801 may generate an access unit delimiter that does not include parameters related to the access unit, instead of an access unit header. This access unit delimiter is used as identification information to indicate the beginning of the access unit. The decoding device identifies the beginning of the access unit by detecting the access unit header or the access unit delimiter.

[0198] Next, the generation of identification information at the beginning of the GOF will be explained. The encoding unit 4801 generates a GOF header as identification information indicating the beginning of the GOF. The encoding unit 4801 stores parameters related to the GOF in the GOF header. For example, the GOF header includes the structure or information of the encoded data included in the GOF. The GOF header also includes parameters commonly used in the data included in the GOF, such as parameters related to decoding the encoded data.

[0199] The encoding unit 4801 may generate a GOF delimiter that does not include parameters related to the GOF, instead of a GOF header. This GOF delimiter is used as identification information to indicate the beginning of the GOF. The decoding device identifies the beginning of the GOF by detecting either the GOF header or the GOF delimiter.

[0200] In PCC encoded data, for example, an access unit is defined as a PCC frame. The decoder accesses the PCC frame based on the identification information at the beginning of the access unit.

[0201] Furthermore, for example, a GOF (Group of Frames) is defined as a single random access unit. The decryption device accesses the random access unit based on the identification information at the beginning of the GOF. For example, if PCC frames are independent of each other and can be decrypted individually, then a PCC frame may be defined as a random access unit.

[0202] Furthermore, two or more PCC frames may be assigned to a single access unit, and multiple random access units may be assigned to a single GOF.

[0203] Furthermore, the encoding unit 4801 may define and generate parameter sets or metadata other than those described above. For example, the encoding unit 4801 may generate SEI (Supplemental Enhancement Information) that stores parameters that may not necessarily be used during decoding (optional parameters).

[0204] Next, we will explain the structure of the encoded data and how to store the encoded data in the NAL unit.

[0205] For example, a data format is defined for each type of encoded data. Figure 26 shows an example of encoded data and a NAL unit.

[0206] For example, as shown in Figure 26, encoded data includes a header and a payload. The encoded data may also include length information indicating the length (data volume) of the encoded data, header, or payload. Furthermore, the encoded data does not necessarily have to include a header.

[0207] The header includes, for example, identification information to identify the data. This identification information may indicate, for example, the data type or frame number.

[0208] The header contains, for example, identification information indicating a reference relationship. This identification information is stored in the header when there is a dependency between data, and it is information used to reference the referenced data from the source. For example, the header of the referenced data contains identification information to identify that data. The header of the referenced data contains identification information indicating the referenced data.

[0209] Furthermore, if the referenced or source can be identified or derived from other information, the identifying information for identifying the data or identifying information indicating the reference relationship may be omitted.

[0210] The multiplexing unit 4802 stores the encoded data in the payload of the NAL unit. The NAL unit header contains pcc_nal_unit_type, which is identification information for the encoded data. Figure 27 shows an example of the semantics of pcc_nal_unit_type.

[0211] As shown in Figure 27, when pcc_codec_type is Codec 1 (Codec1: First encoding method), values ​​of pcc_nal_unit_type from 0 to 10 are assigned to the encoded position data (Geometry), encoded attribute X data (AttributeX), encoded attribute Y data (AttributeY), position PS (Geom.PS), attribute XPS (AttrX.PS), attribute YPS (AttrX.PS), position SPS (Geometry Sequence PS), attribute XSPS (AttributeX Sequence PS), attribute YSPS (AttributeY Sequence PS), AU header (AU Header), and GOF header (GOF Header) in Codec 1. Values ​​11 and above are assigned to the reserves of Codec 1.

[0212] If pcc_codec_type is Codec 2 (the second encoding method), values ​​of pcc_nal_unit_type from 0 to 2 are assigned to the codec's Data A, Metadata A, and Metadata B. Values ​​3 and above are assigned to the backup of Codec 2.

[0213] Next, we will explain the data transmission order. Below, we will explain the constraints on the transmission order of the NAL unit.

[0214] The multiplexing unit 4802 sends out NAL units in groups of GOF or AU units. The multiplexing unit 4802 places a GOF header at the beginning of a GOF and an AU header at the beginning of an AU.

[0215] The multiplexing unit 4802 may provide a sequence parameter set (SPS) for each AU so that the decoding device can decode from the next AU even if data is lost due to packet loss or other reasons.

[0216] If there are dependencies in the encoded data related to decoding, the decoding device decodes the referenced data first, and then decodes the source data. In order for the decoding device to decode the data in the order it was received without rearranging it, the multiplexing unit 4802 sends the referenced data first.

[0217] Figure 28 shows an example of the transmission order of the NAL unit. Figure 28 shows three examples: location information priority, parameter priority, and data integration.

[0218] The location-prioritized transmission order is an example where location information and attribute information are transmitted together. In this transmission order, the transmission of location information is completed earlier than the transmission of attribute information.

[0219] For example, by using this transmission order, a decoding device that does not decode attribute information may be able to create a period of time where it does not process attribute information by ignoring the decoding of attribute information. Also, for example, a decoding device that wants to decode location information quickly may be able to decode the location information faster by obtaining the encoded location information data earlier.

[0220] Note that in Figure 28, the attributes XSPS and YSPS are combined and labeled as attribute SPS, but it is also acceptable to place attributes XSPS and YSPS separately.

[0221] In a parameter set priority transmission order, the parameter set is sent first, followed by the data.

[0222] As long as the constraints on the NAL unit transmission order are followed as described above, the multiplexing unit 4802 may transmit the NAL units in any order. For example, sequence identification information may be defined, and the multiplexing unit 4802 may have the function of transmitting NAL units in multiple patterns of order. For example, the sequence identification information of the NAL units may be stored in the stream PS.

[0223] The three-dimensional data decoding device may perform decoding based on sequence identification information. The three-dimensional data decoding device may instruct the three-dimensional data encoding device to send a desired transmission order, and the three-dimensional data encoding device (multiplexing unit 4802) may control the transmission order according to the instructed transmission order.

[0224] Furthermore, the multiplexing unit 4802 may generate encoded data that merges multiple functions, as long as it adheres to the constraints of the transmission order, such as the transmission order of data integration. For example, as shown in Figure 28, the GOF header and the AU header may be merged, or AXPS and AYPS may be merged. In this case, pcc_nal_unit_type is defined as an identifier indicating that the data has multiple functions.

[0225] The following describes modifications of this embodiment. PS has levels, such as frame-level PS, sequence-level PS, and PCC sequence-level PS. If we consider the PCC sequence level as the higher level and the frame level as the lower level, the following method may be used to store the parameters.

[0226] The default PS value is shown in the higher-level PS. If the value of the lower-level PS differs from the value of the higher-level PS, the PS value is shown in the lower-level PS. Alternatively, the PS value is not listed in the higher-level PS, but is listed in the lower-level PS. Alternatively, information on whether the PS value is shown in the lower-level PS, the higher-level PS, or both is shown in either the lower-level PS or the higher-level PS, or both. Alternatively, the lower-level PS may be merged with the higher-level PS. Alternatively, if the lower-level PS and the higher-level PS overlap, the multiplexing unit 4802 may omit sending one of them.

[0227] The encoding unit 4801 or the multiplexing unit 4802 may divide the data into slices or tiles and send out the divided data. The divided data includes information for identifying the divided data, and the parameter set includes parameters used for decoding the divided data. In this case, pcc_nal_unit_type is defined as an identifier indicating that it is data that stores data or parameters related to tiles or slices.

[0228] The following describes the processing related to sequence identification information. Figure 29 is a flowchart of the processing performed by the three-dimensional data encoding device (encoding unit 4801 and multiplexing unit 4802) related to the transmission order of the NAL unit.

[0229] First, the three-dimensional data encoding device determines the transmission order of the NAL units (position information priority or parameter set priority) (S4801). For example, the three-dimensional data encoding device determines the transmission order based on a specification from the user or an external device (e.g., a three-dimensional data decoding device).

[0230] If the determined transmission order prioritizes location information (location information priority in S4802), the three-dimensional data encoding device sets the sequence identification information included in the stream PS to prioritize location information (S4803). In other words, in this case, the sequence identification information indicates that the NAL units will be transmitted in the order prioritizing location information. The three-dimensional data encoding device then transmits the NAL units in the order prioritizing location information (S4804).

[0231] On the other hand, if the determined transmission order is parameter set priority (parameter set priority in S4802), the three-dimensional data encoding device sets the sequence identification information included in the stream PS to parameter set priority (S4805). In other words, in this case, the sequence identification information indicates that the NAL units will be transmitted in the parameter set priority order. The three-dimensional data encoding device then transmits the NAL units in the parameter set priority order (S4806).

[0232] Figure 30 is a flowchart of the processing performed by the three-dimensional data decoding device regarding the transmission order of the NAL unit. First, the three-dimensional data decoding device analyzes the sequence identification information contained in the stream PS (S4811).

[0233] If the transmission order indicated by the sequence identification information is prioritized by position information (position information priority in S4812), the three-dimensional data decoding device decodes the NAL units assuming that the transmission order of the NAL units is prioritized by position information (S4813).

[0234] On the other hand, if the transmission order indicated by the sequence identification information is parameter set priority (parameter set priority in S4812), the three-dimensional data decoding device decodes the NAL units assuming that the transmission order of the NAL units is parameter set priority (S4814).

[0235] For example, if the three-dimensional data decoding device does not decode attribute information, in step S4813 it may not acquire all NAL units, but instead acquire NAL units related to location information and decode the location information from the acquired NAL units.

[0236] Next, we will explain the process related to the generation of AUs and GOFs. Figure 31 is a flowchart of the processing by the three-dimensional data encoding device (multiplexing unit 4802) related to the generation of AUs and GOFs in the multiplexing of NAL units.

[0237] First, the three-dimensional data encoding device determines the type of encoded data (S4821). Specifically, the three-dimensional data encoding device determines whether the encoded data to be processed is data starting with AU, data starting with GOF, or other data.

[0238] If the encoded data is the first data in the GOF (GOF first in S4822), the three-dimensional data encoding device places the GOF header and AU header at the beginning of the encoded data belonging to the GOF and generates a NAL unit (S4823).

[0239] If the encoded data is the data at the beginning of the AU (AU at the beginning in S4822), the three-dimensional data encoding device places the AU header at the beginning of the encoded data belonging to the AU and generates a NAL unit (S4824).

[0240] If the encoded data is neither GOF-first nor AU-first (other than GOF-first or AU-first in S4822), the three-dimensional data encoding device generates a NAL unit by placing the encoded data after the AU header of the AU to which the encoded data belongs (S4825).

[0241] Next, we will explain the processing related to accessing the AU and GOF. Figure 32 is a flowchart of the processing of the three-dimensional data decoding device related to accessing the AU and GOF in the demultiplexing of the NAL unit.

[0242] First, the three-dimensional data decoding device determines the type of encoded data contained in the NAL unit by analyzing the nal_unit_type contained in the NAL unit (S4831). Specifically, the three-dimensional data decoding device determines whether the encoded data contained in the NAL unit is data starting with AU, data starting with GOF, or other data.

[0243] If the encoded data contained in the NAL unit is the data at the beginning of the GOF (the beginning of the GOF in S4832), the three-dimensional data decoding device determines that the NAL unit is the starting position for random access, accesses the NAL unit, and starts the decoding process (S4833).

[0244] On the other hand, if the encoded data contained in the NAL unit is the data at the beginning of the AU (AU at the beginning in S4832), the three-dimensional data decoding device determines that the NAL unit is at the beginning of the AU, accesses the data contained in the NAL unit, and decodes the AU (S4834).

[0245] On the other hand, if the encoded data contained in the NAL unit is neither GOF-first nor AU-first (other than GOF-first or AU-first in S4832), the three-dimensional data decoding device does not process the NAL unit.

[0246] (Embodiment 3) This embodiment describes a method for representing three-dimensional points (point clouds) in the encoding of three-dimensional data.

[0247] Figure 33 is a block diagram showing the configuration of a three-dimensional data distribution system according to this embodiment. The distribution system shown in Figure 33 includes a server 1501 and a plurality of clients 1502.

[0248] The server 1501 includes a storage unit 1511 and a control unit 1512. The storage unit 1511 stores an encoded three-dimensional map 1513, which is encoded three-dimensional data.

[0249] Figure 34 shows an example of the bitstream configuration of the encoded three-dimensional map 1513. The three-dimensional map is divided into multiple sub-maps, and each sub-map is encoded. Each sub-map is accompanied by a random access header (RA) containing sub-coordinate information. The sub-coordinate information is used to improve the encoding efficiency of the sub-maps. This sub-coordinate information indicates the sub-coordinate of the sub-map. The sub-coordinate is the coordinate of the sub-map relative to the reference coordinate. A three-dimensional map containing multiple sub-maps is called the overall map. The coordinate that serves as the reference in the overall map (e.g., the origin) is called the reference coordinate. In other words, the sub-coordinate is the coordinate of the sub-map in the coordinate system of the overall map. To put it another way, the sub-coordinate indicates the offset between the coordinate system of the overall map and the coordinate system of the sub-map. The coordinate in the coordinate system of the overall map relative to the reference coordinate is called the overall coordinate. The coordinate in the coordinate system of the sub-map relative to the sub-coordinate is called the difference coordinate.

[0250] Client 1502 sends a message to Server 1501. This message includes the location information of Client 1502. The control unit 1512 in Server 1501 obtains a bitstream of the submap closest to Client 1502's location based on the location information contained in the received message. The bitstream of the submap includes sub-coordinate information and is sent to Client 1502. The decoder 1521 in Client 1502 uses this sub-coordinate information to obtain the overall coordinates of the submap relative to the reference coordinates. The application 1522 in Client 1502 uses the obtained overall coordinates of the submap to execute an application related to its own location.

[0251] Furthermore, a submap represents a portion of the overall map. The sub-coordinates are the coordinates in which the submap is located within the reference coordinate space of the overall map. For example, suppose that within the overall map A, there is submap A of AA and submap B of AB. If the vehicle wants to refer to the map of AA, it will start decoding from submap A, and if it wants to refer to the map of AB, it will start decoding from submap B. Here, the submaps are random access points. Specifically, A is Osaka Prefecture, AA is Osaka City, AB is Takatsuki City, and so on.

[0252] Each submap is sent to the client along with its sub-coordinate information. This sub-coordinate information is included in the header information of each submap or in the transmitted packet.

[0253] The reference coordinates that serve as the basis for the sub-coordinate information of each submap may be added to the header information of a higher-level space than the submap, such as the header information of the overall map.

[0254] A submap may consist of one space (SPC). Alternatively, a submap may consist of multiple SPCs.

[0255] Furthermore, a submap may include a Group of Space (GOS). A submap may also consist of worlds. For example, if a submap contains multiple objects, assigning those objects to separate SPCs will result in the submap being composed of multiple SPCs. Similarly, assigning multiple objects to a single SPC will result in the submap being composed of a single SPC.

[0256] Next, we will explain the improvement in encoding efficiency when using sub-coordinate information. Figure 35 is a diagram illustrating this effect. For example, as shown in Figure 35, encoding a three-dimensional point A located far from the reference coordinates requires a large number of bits. Here, the distance between the sub-coordinates and three-dimensional point A is shorter than the distance between the reference coordinates and three-dimensional point A. Therefore, encoding efficiency can be improved by encoding the coordinates of three-dimensional point A based on the sub-coordinates rather than encoding the coordinates of three-dimensional point A based on the reference coordinates. In addition, the bitstream of the submap contains sub-coordinate information. By sending the bitstream of the submap and the reference coordinates to the decoding side (client), the entire coordinates of the submap can be reconstructed on the decoding side.

[0257] Figure 36 is a flowchart of the processing performed by server 1501, which is the sender of the submap.

[0258] First, server 1501 receives a message from client 1502 containing the client's location information (S1501). Control unit 1512 obtains an encoded bitstream of a submap based on the client's location information from storage unit 1511 (S1502). Then, server 1501 transmits the encoded bitstream of the submap and the reference coordinates to client 1502 (S1503).

[0259] Figure 37 is a flowchart of the processing performed by client 1502, which is the recipient of the submap.

[0260] First, client 1502 receives the encoded bitstream of the submap and the reference coordinates transmitted from server 1501 (S1511). Next, client 1502 decodes the encoded bitstream to obtain the submap and sub-coordinate information (S1512). Then, client 1502 uses the reference coordinates and sub-coordinates to reconstruct the difference coordinates within the submap into the overall coordinates (S1513).

[0261] Next, we will explain an example of the syntax for information about submaps. In encoding submaps, the 3D data encoding device calculates the difference coordinates by subtracting the sub-coordinates from the coordinates of each point cloud (3D point). The 3D data encoding device then encodes the difference coordinates into a bitstream as the value for each point cloud. The encoding device also encodes the sub-coordinate information, which indicates the sub-coordinates, as header information for the bitstream. This allows the 3D data decoding device to obtain the overall coordinates of each point cloud. For example, the 3D data encoding device is included in server 1501, and the 3D data decoding device is included in client 1502.

[0262] Figure 38 shows an example of submap syntax. NumOfPoint in Figure 38 indicates the number of point clouds included in the submap. sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z are sub-coordinate information. sub_coordinate_x indicates the x-coordinate of the sub-coordinate. sub_coordinate_y indicates the y-coordinate of the sub-coordinate. sub_coordinate_z indicates the z-coordinate of the sub-coordinate.

[0263] Furthermore, diff_x[i], diff_y[i], and diff_z[i] are the difference coordinates of the i-th point cloud in the submap. diff_x[i] shows the difference between the x-coordinate of the i-th point cloud in the submap and the x-coordinate of the submap. diff_y[i] shows the difference between the y-coordinate of the i-th point cloud in the submap and the y-coordinate of the submap. diff_z[i] shows the difference between the z-coordinate of the i-th point cloud in the submap and the z-coordinate of the submap.

[0264] The three-dimensional data decoding device decodes the global coordinates of the i-th point cloud, point_cloud[i]_x, point_cloud[i]_y, and point_cloud[i]_z, using the following formulas: point_cloud[i]_x is the x-coordinate of the global coordinates of the i-th point cloud. point_cloud[i]_y is the y-coordinate of the global coordinates of the i-th point cloud. point_cloud[i]_z is the z-coordinate of the global coordinates of the i-th point cloud.

[0265] point_cloud[i]_x=sub_coordinate_x+diff_x[i] point_cloud[i]_y=sub_coordinate_y+diff_y[i] point_cloud[i]_z=sub_coordinate_z+diff_z[i]

[0266] Next, we will explain the switching process for applying octree coding. When coding a submap, the three-dimensional data coding device selects whether to encode each point cloud using an octree representation (hereinafter referred to as octree coding) or to encode the difference value from the sub-coordinates (hereinafter referred to as non-octree coding). Figure 39 schematically illustrates this operation. For example, if the number of point clouds in the submap is greater than or equal to a predetermined threshold, the three-dimensional data coding device applies octree coding to the submap. If the number of point clouds in the submap is less than the threshold, the three-dimensional data coding device applies non-octree coding to the submap. This allows the three-dimensional data coding device to appropriately select whether to use octree coding or non-octree coding depending on the shape and density of the objects contained in the submap, thereby improving coding efficiency.

[0267] Furthermore, the three-dimensional data encoding device adds information to the submap header, etc., indicating whether octree encoding or non-octree encoding was applied to the submap (hereinafter referred to as octree encoding application information). This allows the three-dimensional data decoding device to determine whether the bitstream is a bitstream obtained by octree encoding of the submap or a bitstream obtained by non-octree encoding of the submap.

[0268] Furthermore, the three-dimensional data encoding device may calculate the encoding efficiency when applying both octree encoding and non-octree encoding to the same point cloud, and then apply the encoding method with the best encoding efficiency to the submap.

[0269] Figure 40 shows an example of the submap syntax when this switching is performed. The coding_type shown in Figure 40 is information indicating the encoding type, and is the octave coding application information described above. coding_type=00 indicates that octave coding has been applied. coding_type=01 indicates that non-octave coding has been applied. coding_type=10 or 11 indicates that other encoding schemes not mentioned above have been applied.

[0270] If the encoding type is non-octree encoding, the submap contains NumOfPoint and subcoordinate information (sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z).

[0271] If the encoding type is octree coding, the submap includes octree_info. octree_info contains information necessary for octree coding, such as depth information.

[0272] If the encoding type is non-octree encoding, the submap includes the difference coordinates (diff_x[i], diff_y[i], and diff_z[i]).

[0273] If the encoding type is octree coding, the submap contains octree_data, which is the encoded data related to the octree coding.

[0274] In this example, the xyz coordinate system is used as the coordinate system for the point cloud, but the polar coordinate system may also be used.

[0275] Figure 41 is a flowchart of the three-dimensional data encoding process performed by the three-dimensional data encoding device. First, the three-dimensional data encoding device calculates the number of point clouds within the target submap, which is the submap to be processed (S1521). Next, the three-dimensional data encoding device determines whether the calculated number of point clouds is equal to or greater than a predetermined threshold (S1522).

[0276] If the number of point clouds is greater than or equal to a threshold (Yes in S1522), the three-dimensional data encoding device applies octree encoding to the target submap (S1523). The three-dimensional point data encoding device also adds octree encoding application information to the bitstream header to indicate that octree encoding has been applied to the target submap (S1525).

[0277] On the other hand, if the number of point clouds is less than the threshold (No in S1522), the three-dimensional data encoding device applies non-octree encoding to the target submap (S1524). The three-dimensional point data encoding device also adds octree encoding application information to the bitstream header to indicate that non-octree encoding has been applied to the target submap (S1525).

[0278] Figure 42 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding device. First, the three-dimensional data decoding device decodes the octave coding application information from the bitstream header (S1531). Next, the three-dimensional data decoding device determines whether the coding type applied to the target submap is octave coding or not based on the decoded octave coding application information (S1532).

[0279] If the encoding type indicated by the octree encoding application information is octree encoding (Yes in S1532), the three-dimensional data decoder decodes the target submap by octree decoding (S1533). On the other hand, if the encoding type indicated by the octree encoding application information is not octree encoding (No in S1532), the three-dimensional data decoder decodes the target submap by non-octree decoding (S1534).

[0280] The following describes modifications of this embodiment. Figures 43 to 45 schematically show the operation of a modified version of the encoding type switching process.

[0281] As shown in Figure 43, the three-dimensional data encoding device may choose on a space-by-space basis whether to apply octree encoding or non-octree encoding. In this case, the three-dimensional data encoding device adds octree encoding application information to the space header. This allows the three-dimensional data decoding device to determine on a space-by-space basis whether octree encoding has been applied or not. In this case, the three-dimensional data encoding device also sets sub-coordinates for each space and encodes the difference obtained by subtracting the sub-coordinate value from the coordinates of each point cloud within the space.

[0282] This allows the three-dimensional data encoding device to appropriately switch whether or not to apply octave coding depending on the shape of the objects in the space or the number of point clouds, thereby improving encoding efficiency.

[0283] Furthermore, as shown in Figure 44, the three-dimensional data encoding device may select whether to apply octree encoding or non-octree encoding on a volume-by-volume basis. In this case, the three-dimensional data encoding device adds octree encoding application information to the volume header. This allows the three-dimensional data decoding device to determine whether octree encoding has been applied to each volume. In this case, the three-dimensional data encoding device sets sub-coordinates for each volume and encodes the difference obtained by subtracting the sub-coordinate values ​​from the coordinates of each point cloud within the volume.

[0284] This allows the three-dimensional data encoding device to appropriately switch whether or not to apply octave coding depending on the shape of the objects in the volume or the number of point clouds, thereby improving encoding efficiency.

[0285] Furthermore, while the above explanation showed an example of encoding the difference obtained by subtracting the sub-coordinates from the coordinates of each point cloud as a non-octree coding method, it is not necessarily limited to this, and any coding method other than octree coding may be used. For example, as shown in Figure 45, the three-dimensional data coding device may use a method that encodes the value of the point cloud itself within the submap, space, or volume, rather than the difference from the sub-coordinates, as a non-octree coding method (hereinafter referred to as original coordinate coding).

[0286] In that case, the three-dimensional data encoding device stores information in the header indicating that original coordinate encoding has been applied to the target space (submap, space, or volume). This allows the three-dimensional data decoding device to determine whether or not original coordinate encoding has been applied to the target space.

[0287] Furthermore, when applying original coordinate coding, the three-dimensional data coding device may perform coding without applying quantization and arithmetic coding to the original coordinates. Alternatively, the three-dimensional data coding device may code the original coordinates with a predetermined fixed bit length. This allows the three-dimensional data coding device to generate a stream of a fixed bit length at a given time.

[0288] Furthermore, the above explanation showed an example of non-octave tree coding where the difference obtained by subtracting the sub-coordinates from the coordinates of each point cloud is encoded, but this is not necessarily the only method.

[0289] For example, the three-dimensional data encoding device may sequentially encode the difference values ​​between the coordinates of each point cloud. Figure 46 is a diagram illustrating this operation. For example, in the example shown in Figure 46, when encoding point cloud PA, the three-dimensional data encoding device uses the sub-coordinates as predicted coordinates and encodes the difference value between the coordinates of point cloud PA and the predicted coordinates. Also, when encoding point cloud PB, the three-dimensional data encoding device uses the coordinates of point cloud PA as predicted coordinates and encodes the difference value between point cloud PB and the predicted coordinates. Furthermore, when encoding point cloud PC, the three-dimensional data encoding device uses point cloud PB as predicted coordinates and encodes the difference value between point cloud PB and the predicted coordinates. In this way, the three-dimensional data encoding device may set a scan order for multiple point clouds and encode the difference value between the coordinates of the target point cloud to be processed and the coordinates of the point cloud immediately preceding the target point cloud in the scan order.

[0290] Furthermore, in the above explanation, the sub-coordinate was the coordinate of the lower left front corner of the submap, but the position of the sub-coordinate is not limited to this. Figures 47 to 49 show other examples of the position of the sub-coordinate. The sub-coordinate can be set at any coordinate within the target space (submap, space, or volume). In other words, as mentioned above, the sub-coordinate may be the coordinate of the lower left front corner of the target space. As shown in Figure 47, the sub-coordinate may be the coordinate of the center of the target space. As shown in Figure 48, the sub-coordinate may be the coordinate of the upper right rear corner of the target space. Moreover, the sub-coordinate is not limited to the lower left front or upper right rear corner of the target space, but may be the coordinate of any corner of the target space.

[0291] Furthermore, the location of the sub-coordinates may be the same as the coordinates of a point cloud within the target space (submap, space, or volume). For example, in the example shown in Figure 49, the sub-coordinates coincide with the coordinates of the point cloud PD.

[0292] Furthermore, while this embodiment demonstrates an example of switching between applying octree coding and non-octree coding, it is not necessarily limited to this. For example, the three-dimensional data encoding device may switch between applying a tree structure other than an octree and applying a non-tree structure other than the octree structure. For example, the other tree structure could be a kd-tree that performs partitioning using a plane perpendicular to one of the coordinate axes. Any method can be used as the other tree structure.

[0293] Furthermore, while this embodiment demonstrates an example of encoding the coordinate information of a point cloud, it is not necessarily limited to this. The three-dimensional data encoding device may encode, for example, color information, three-dimensional features, or visible light features in the same way as the coordinate information. For example, the three-dimensional data encoding device may set the average value of the color information of each point cloud in the submap as sub-color information, and encode the difference between the color information of each point cloud and the sub-color information.

[0294] Furthermore, while this embodiment shows an example of selecting an encoding method (octree encoding or non-octree encoding) that offers good encoding efficiency depending on the number of point clouds, etc., it is not necessarily limited to this. For example, the server-side three-dimensional data encoding device may store bitstreams of point clouds encoded by octree encoding, bitstreams of point clouds encoded by non-octree encoding, and bitstreams of point clouds encoded by both, and switch the bitstreams to send to the three-dimensional data decoding device depending on the communication environment or the processing capacity of the three-dimensional data decoding device.

[0295] Figure 50 shows an example of volume syntax when switching the application of octree coding. The syntax shown in Figure 50 is basically the same as the syntax shown in Figure 40, but differs in that each piece of information is volume-level information. Specifically, NumOfPoint indicates the number of point clouds contained in the volume. sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z are sub-coordinate information of the volume.

[0296] Furthermore, diff_x[i], diff_y[i], and diff_z[i] are the difference coordinates of the i-th point cloud within the volume. diff_x[i] shows the difference between the x-coordinate of the i-th point cloud within the volume and the x-coordinate of the sub-coordinate. diff_y[i] shows the difference between the y-coordinate of the i-th point cloud within the volume and the y-coordinate of the sub-coordinate. diff_z[i] shows the difference between the z-coordinate of the i-th point cloud within the volume and the z-coordinate of the sub-coordinate.

[0297] Furthermore, if the relative position of the volume in space can be calculated, the 3D data encoding device does not need to include sub-coordinate information in the volume header. In other words, the 3D data encoding device may calculate the relative position of the volume in space without including sub-coordinate information in the header and use the calculated position as the sub-coordinate of each volume.

[0298] As described above, the three-dimensional data encoding device according to this embodiment determines whether or not to encode a target spatial unit (e.g., a submap, space, or volume) included in the three-dimensional data using an octave structure (for example, S1522 in Figure 41). For example, the three-dimensional data encoding device determines to encode the target spatial unit using an octave structure if the number of three-dimensional points included in the target spatial unit exceeds a predetermined threshold. Conversely, the three-dimensional data encoding device determines not to encode the target spatial unit using an octave structure if the number of three-dimensional points included in the target spatial unit is less than or equal to the threshold.

[0299] If it is determined that the target spatial unit should be encoded using an octave tree structure (Yes in S1522), the three-dimensional data encoding device encodes the target spatial unit using an octave tree structure (S1523). If it is determined that the target spatial unit should not be encoded using an octave tree structure (No in S1522), the three-dimensional data encoding device encodes the target spatial unit using a method different from the octave tree structure (S1524). For example, in the different method, the three-dimensional data encoding device encodes the coordinates of the three-dimensional points contained within the target spatial unit. Specifically, in the different method, the three-dimensional data encoding device encodes the difference between the reference coordinates of the target spatial unit and the coordinates of the three-dimensional points contained within the target spatial unit.

[0300] Next, the three-dimensional data encoding device adds information to the bitstream indicating whether or not the target spatial unit has been encoded in an octave tree structure (S1525).

[0301] According to this, the three-dimensional data encoding device can reduce the amount of data in the encoded signal, thereby improving encoding efficiency.

[0302] For example, a three-dimensional data encoding device comprises a processor and memory, and the processor uses the memory to perform the above processing.

[0303] Furthermore, the three-dimensional data decoding device according to this embodiment decodes from the bitstream information indicating whether or not to decode a target spatial unit (e.g., a submap, space, or volume) included in the three-dimensional data using an octave structure (for example, S1531 in Figure 42). If the above information indicates that the target spatial unit should be decoded using an octave structure (Yes in S1532), the three-dimensional data decoding device decodes the target spatial unit using an octave structure (S1533).

[0304] If the above information indicates that the target spatial unit will not be decoded using an octree structure (No in S1532), the three-dimensional data decoding device decodes the target spatial unit using a method different from the octree structure (S1534). For example, in the different method, the three-dimensional data decoding device decodes the coordinates of the three-dimensional points contained in the target spatial unit. Specifically, in the different method, the three-dimensional data decoding device decodes the difference between the reference coordinates of the target spatial unit and the coordinates of the three-dimensional points contained in the target spatial unit.

[0305] According to this, the three-dimensional data decoding device can reduce the amount of data in the encoded signal, thereby improving encoding efficiency.

[0306] For example, a three-dimensional data decoding device comprises a processor and memory, and the processor uses the memory to perform the above processing.

[0307] (Embodiment 4) Three-dimensional point cloud information includes geometry and attribute information. Geometry includes coordinates (x, y, and z coordinates) relative to a given point. When encoding geometry, instead of directly encoding the coordinates of each three-dimensional point, a method is used to reduce the amount of encoding by representing the position of each three-dimensional point using an octave tree and encoding the information in the octave tree.

[0308] On the other hand, attribute information includes information such as color information (RGB, YUV, etc.), reflectance, and normal vector for each three-dimensional point. For example, a three-dimensional data encoding device can encode attribute information using a different encoding method than that used for positional information.

[0309] This embodiment describes a method for encoding attribute information. In this embodiment, integer values ​​are used as the values ​​of the attribute information. For example, if each color component of the RGB or YUV color information is 8-bit precision, each color component can take an integer value between 0 and 255. If the reflectance value is 10-bit precision, the reflectance value can take an integer value between 0 and 1023. If the bit precision of the attribute information is decimal precision, the three-dimensional data encoding device may multiply the attribute information value by a scale value and then round it to an integer value. The three-dimensional data encoding device may also add this scale value to the bitstream header, etc.

[0310] One possible method for encoding attribute information of a three-dimensional point is to calculate a predicted value for the attribute information of the three-dimensional point and encode the difference (prediction residual) between the original attribute information value and the predicted value. For example, if the attribute information value of a three-dimensional point p is Ap and the predicted value is Pp, the three-dimensional data encoding device encodes the absolute difference Diffp = |Ap - Pp|. In this case, if the predicted value Pp can be generated with high accuracy, the value of the absolute difference Diffp will become smaller. Therefore, for example, the amount of encoding can be reduced by entropy encoding the absolute difference Diffp using an encoding table where the number of generated bits decreases as the value becomes smaller.

[0311] One possible method for generating predicted attribute information is to use the attribute information of a reference three-dimensional point, which is another three-dimensional point located around the target three-dimensional point to be encoded. Here, a reference three-dimensional point is a three-dimensional point located within a predetermined distance range from the target three-dimensional point. For example, if there is a target three-dimensional point p=(x1,y1,z1) and a three-dimensional point q=(x2,y2,z2), the three-dimensional data encoding device calculates the Euclidean distance d(p,q) between the three-dimensional point p and the three-dimensional point q as shown in (Equation A1).

[0312]

number

[0313] The 3D data encoding device determines that the position of 3D point q is close to the position of target 3D point p if the Euclidean distance d(p, q) is smaller than a predetermined threshold THd, and decides to use the attribute information value of 3D point q to generate the predicted attribute information value of target 3D point p. Note that the distance calculation method may be other; for example, the Mahalanobis distance may be used. Furthermore, the 3D data encoding device may decide not to use 3D points outside a predetermined distance range from the target 3D point in the prediction process. For example, if a 3D point r exists and the distance d(p, r) between target 3D p and 3D point r is greater than or equal to the threshold THd, the 3D data encoding device may decide not to use 3D point r in the prediction. Note that the 3D data encoding device may add information indicating the threshold THd to the bitstream header, etc.

[0314] Figure 51 shows an example of a three-dimensional point. In this example, the distance d(p, q) between the target three-dimensional point p and the three-dimensional point q is smaller than the threshold THd. Therefore, the three-dimensional data encoding device determines that the three-dimensional point q is the reference three-dimensional point of the target three-dimensional point p, and decides to use the value of the attribute information Aq of the three-dimensional point q to generate the predicted value Pp of the attribute information Ap of the target three-dimensional point p.

[0315] On the other hand, the distance d(p,r) between the target three-dimensional point p and the three-dimensional point r is greater than or equal to the threshold THd. Therefore, the three-dimensional data encoding device determines that the three-dimensional point r is not a reference three-dimensional point of the target three-dimensional point p, and determines that it will not use the value of the attribute information Ar of the three-dimensional point r to generate the predicted value Pp of the attribute information Ap of the target three-dimensional point p.

[0316] Furthermore, when a three-dimensional data encoding device encodes the attribute information of a target three-dimensional point using predicted values, it uses a three-dimensional point whose attribute information has already been encoded and decoded as a reference three-dimensional point. Similarly, when a three-dimensional data decoding device decodes the attribute information of a target three-dimensional point to be decoded using predicted values, it uses a three-dimensional point whose attribute information has already been decoded as a reference three-dimensional point. This allows the same predicted values ​​to be generated during encoding and decoding, so that the bitstream of three-dimensional points generated during encoding can be correctly decoded on the decoding side.

[0317] Furthermore, when encoding the attribute information of three-dimensional points, it is conceivable to classify each three-dimensional point into multiple levels using its positional information before encoding. Here, each classified level is called LoD (Level of Detail). The method for generating LoDs will be explained using Figure 52.

[0318] First, the 3D data encoding device selects an initial point a0 and assigns it to LoD0. Next, the 3D data encoding device extracts point a1 whose distance from point a0 is greater than the LoD0 threshold Thres_LoD[0] and assigns it to LoD0. Next, the 3D data encoding device extracts point a2 whose distance from point a1 is greater than the LoD0 threshold Thres_LoD[0] and assigns it to LoD0. In this way, the 3D data encoding device configures LoD0 such that the distance between each point in LoD0 is greater than the threshold Thres_LoD[0].

[0319] Next, the 3D data encoding device selects point b0, which has not yet been assigned a Level of Direction (LoD), and assigns it to LoD1. Next, the 3D data encoding device extracts point b1, which is farther from point b0 than the LoD1 threshold Thres_LoD[1] and has not yet been assigned a Level of Direction (LoD), and assigns it to LoD1. Next, the 3D data encoding device extracts point b2, which is farther from point b1 than the LoD1 threshold Thres_LoD[1] and has not yet been assigned a Level of Direction (LoD), and assigns it to LoD1. In this way, the 3D data encoding device configures LoD1 such that the distance between each point in LoD1 is greater than the threshold Thres_LoD[1].

[0320] Next, the 3D data encoding device selects point c0, which has not yet been assigned a LoD, and assigns it to LoD2. Next, the 3D data encoding device extracts point c1, which is farther from point c0 than the LoD2 threshold Thres_LoD[2] and has not yet been assigned a LoD, and assigns it to LoD2. Next, the 3D data encoding device extracts point c2, which is farther from point c1 than the LoD2 threshold Thres_LoD[2] and has not yet been assigned a LoD, and assigns it to LoD2. In this way, the 3D data encoding device configures LoD2 such that the distance between each point in LoD2 is greater than the threshold Thres_LoD[2]. For example, as shown in Figure 53, the thresholds Thres_LoD[0], Thres_LoD[1], and Thres_LoD[2] for each LoD are set.

[0321] Furthermore, the three-dimensional data encoding device may add information indicating the threshold for each LoD to the bitstream header, etc. For example, in the example shown in Figure 53, the three-dimensional data encoding device may add the thresholds Thres_LoD[0], Thres_LoD[1], and Thres_LoD[2] to the header.

[0322] Furthermore, the three-dimensional data encoding device may assign all three-dimensional points that have not yet been assigned a LoD to the lowest layer of the LoD. In this case, the three-dimensional data encoding device can reduce the amount of code in the header by not adding the threshold of the lowest layer of the LoD to the header. For example, in the example shown in Figure 53, the three-dimensional data encoding device adds the thresholds Thres_LoD[0] and Thres_LoD[1] to the header, but does not add Thres_LoD[2] to the header. In this case, the three-dimensional data decoding device may estimate the value of Thres_LoD[2] to be 0. The three-dimensional data encoding device may also add the number of LoD layers to the header. This allows the three-dimensional data decoding device to determine the lowest layer of the LoD using the number of LoD layers.

[0323] Furthermore, by setting the threshold values ​​for each layer of the LoD to be larger for higher layers, as shown in Figure 53, higher layers (layers closer to LoD0) become sparse point groups with greater distances between 3D points, while lower layers become dense point groups with closer distances between 3D points. In the example shown in Figure 53, LoD0 is the top layer.

[0324] Furthermore, the method for selecting the initial three-dimensional points when setting each LoD may depend on the coding order during positional information coding. For example, the three-dimensional data encoding device selects the first three-dimensional point coded during positional information coding as the initial point a0 of LoD0, and then uses initial point a0 as the base point to select points a1 and a2 to construct LoD0. Then, the three-dimensional data encoding device may select the three-dimensional point that was coded earliest among the three-dimensional points not belonging to LoD0 as the initial point b0 of LoD1. In other words, the three-dimensional data encoding device may select the three-dimensional point that was coded earliest among the three-dimensional points not belonging to the upper layers of LoDn (LoD0 to LoDn-1) as the initial point n0 of LoDn. As a result, the three-dimensional data decoding device can construct the same LoD as during coding by using the same initial point selection method during decoding, and thus can decode the bitstream appropriately. Specifically, the three-dimensional data decoding device selects the three-dimensional point that was coded earliest among the three-dimensional points not belonging to the upper layers of LoDn as the initial point n0 of LoDn.

[0325] The following describes a method for generating predicted attribute information of three-dimensional points using LoD information. For example, when a three-dimensional data encoding device encodes three-dimensional points sequentially starting from those contained in LoD0, it generates the target three-dimensional points contained in LoD1 using the encoded and decoded (hereinafter simply referred to as "encoded") attribute information contained in LoD0 and LoD1. In this way, the three-dimensional data encoding device generates predicted attribute information of three-dimensional points contained in LoDn using the encoded attribute information contained in LoDn' (n'<=n). In other words, the three-dimensional data encoding device does not use the attribute information of three-dimensional points contained in lower layers of LoDn to calculate predicted attribute information of three-dimensional points contained in LoDn.

[0326] For example, a three-dimensional data encoding device generates predicted attribute values ​​for a three-dimensional point by calculating the average of the attribute values ​​of N or fewer encoded three-dimensional points surrounding the target three-dimensional point to be encoded. Alternatively, the three-dimensional data encoding device may add the value of N to the bitstream header, etc. Furthermore, the three-dimensional data encoding device may change the value of N for each three-dimensional point and add the value of N to each three-dimensional point. This allows for the selection of an appropriate N for each three-dimensional point, thereby improving the accuracy of the predicted values ​​and reducing the prediction residual. Alternatively, the three-dimensional data encoding device may add the value of N to the bitstream header and fix the value of N within the bitstream. This eliminates the need to encode or decode the value of N for each three-dimensional point, thus reducing processing load. Finally, the three-dimensional data encoding device may encode the value of N separately for each Level of Data (LoD). This allows for the selection of an appropriate N for each LoD, improving encoding efficiency.

[0327] Alternatively, the three-dimensional data encoding device may calculate the predicted value of the attribute information of a three-dimensional point by using the weighted average of the attribute information of N surrounding encoded three-dimensional points. For example, the three-dimensional data encoding device calculates weights using the distance information between the target three-dimensional point and the N surrounding three-dimensional points.

[0328] When a 3D data encoding device encodes the value of N separately for each Level of Data (LoD), for example, it may set a larger value of N for higher layers of the LoD and a smaller value of N for lower layers. In the higher layers of the LoD, the distance between the 3D points is greater, so setting a larger value of N and selecting multiple surrounding 3D points for averaging may improve prediction accuracy. Conversely, in the lower layers of the LoD, the distance between the 3D points is smaller, so setting a smaller value of N allows for efficient prediction while reducing the processing load of averaging.

[0329] Figure 54 shows an example of attribute information used for prediction values. As described above, the prediction value of point P included in LoDN is generated using the encoded surrounding points P' included in LoDN'(N'<=N). Here, the surrounding points P' are selected based on their distance from point P. For example, the prediction value of the attribute information of point b2 shown in Figure 54 is generated using the attribute information of points a0, a1, a2, b0, and b1.

[0330] The surrounding points selected change depending on the value of N mentioned above. For example, if N=5, a0, a1, a2, b0, and b1 are selected as surrounding points of point b2. If N=4, points a0, a1, a2, and b1 are selected based on distance information.

[0331] The predicted value is calculated using a distance-dependent weighted average. For example, in the example shown in Figure 54, the predicted value a2p for point a2 is calculated using a weighted average of the attribute information of points a0 and a1, as shown in (Equations A2) and (Equations A3). i This is the attribute information value of point ai.

[0332]

number

[0333] Furthermore, the predicted value b2p for point b2 is calculated by the weighted average of the attribute information of points a0, a1, a2, b0, and b1, as shown in (Equations A4) to (Equations A6). i This is the attribute information value of point bi.

[0334]

number

[0335] Furthermore, the three-dimensional data encoding device may calculate the difference (prediction residual) between the attribute information value of a three-dimensional point and the predicted value generated from surrounding points, and then quantize the calculated prediction residual. For example, the three-dimensional data encoding device performs quantization by dividing the prediction residual by a quantization scale (also called a quantization step). In this case, the smaller the quantization scale, the smaller the error that may occur due to quantization (quantization error). Conversely, the larger the quantization scale, the larger the quantization error.

[0336] Furthermore, the 3D data encoding device may change the quantization scale used for each Level of Data (LoD). For example, the 3D data encoding device may use a smaller quantization scale in the upper layers and a larger quantization scale in the lower layers. Since the attribute information values ​​of 3D points belonging to the upper layers may be used as predicted values ​​for the attribute information of 3D points belonging to the lower layers, encoding efficiency can be improved by reducing the quantization scale in the upper layers to suppress quantization errors that may occur in the upper layers and increasing the accuracy of the predicted values. The 3D data encoding device may also add the quantization scale used for each LoD to the header or other components. This allows the 3D data decoding device to correctly decode the quantization scale, thus enabling proper decoding of the bitstream.

[0337] Furthermore, the three-dimensional data encoding device may convert the signed integer value (signed quantized value), which is the prediction residual after quantization, into an unsigned integer value (unsigned quantized value). This eliminates the need to consider the occurrence of negative integers when entropy encoding the prediction residual. Note that the three-dimensional data encoding device does not necessarily need to convert the signed integer value into an unsigned integer value; for example, the sign bit may be entropy encoded separately.

[0338] The predicted residual is calculated by subtracting the predicted value from the original value. For example, the predicted residual a2r for point a2 is calculated by subtracting the predicted value a2p for point a2 from the attribute information value A2 for point a2, as shown in (Equation A7). The predicted residual b2r for point b2 is calculated by subtracting the predicted value b2p for point b2 from the attribute information value B2 for point b2, as shown in (Equation A8).

[0339] a²r = A² - a²p ... (Equation A7) b²r = B² - b²p ... (Equation A8)

[0340] Furthermore, the predicted residuals are quantized by dividing them by the QS (Quantization Step). For example, the quantized value a2q of point a2 is calculated by (Equation A9). The quantized value b2q of point b2 is calculated by (Equation A10). Here, QS_LoD0 is the QS for LoD0, and QS_LoD1 is the QS for LoD1. In other words, the QS may be changed according to the LoD.

[0341] a2q = a2r / QS_LoD0 ... (Equation A9) b2q=b2r / QS_LoD1 (Formula A10)

[0342] Furthermore, the three-dimensional data encoding device converts the quantized value, which is a signed integer, into an unsigned integer as follows: If the signed integer value a2q is less than 0, the three-dimensional data encoding device sets the unsigned integer value a2u to -1-(2×a2q). If the signed integer value a2q is 0 or greater, the three-dimensional data encoding device sets the unsigned integer value a2u to 2×a2q.

[0343] Similarly, the three-dimensional data encoding device sets the unsigned integer value b2u to -1-(2×b2q) if the signed integer value b2q is less than 0. The three-dimensional data encoding device sets the unsigned integer value b2u to 2×b2q if the signed integer value b2q is 0 or greater.

[0344] Furthermore, the three-dimensional data encoding device may encode the predicted residuals (unsigned integer values) after quantization using entropy coding. For example, the unsigned integer values ​​may be binarized and then binary arithmetic coding may be applied.

[0345] In this case, the three-dimensional data encoding device may switch the binarization method depending on the value of the predicted residual. For example, if the predicted residual pu is smaller than the threshold R_TH, the three-dimensional data encoding device binarizes the predicted residual pu with a fixed number of bits required to represent the threshold R_TH. If the predicted residual pu is greater than or equal to the threshold R_TH, the three-dimensional data encoding device binarizes the binarized data of the threshold R_TH and the value of (pu-R_TH) using an exponential Golomb or the like.

[0346] For example, if the threshold R_TH is 63 and the predicted residual pu is less than 63, the three-dimensional data encoding device will binarize the predicted residual pu using 6 bits. If the predicted residual pu is 63 or greater, the three-dimensional data encoding device will perform arithmetic encoding by binarizing the binary data of the threshold R_TH (111111) and (pu-63) using an exponential golomb.

[0347] In a more specific example, if the predicted residual pu is 32, the three-dimensional data encoding device generates 6 bits of binary data (100000) and arithmetically encodes this bit sequence. Similarly, if the predicted residual pu is 66, the three-dimensional data encoding device generates binary data of the threshold R_TH (111111) and a bit sequence (00100) representing the value 3 (66-63) in exponential golombs, and arithmetically encodes this bit sequence (111111+00100).

[0348] In this way, the three-dimensional data encoding device can encode data while suppressing a rapid increase in the number of binarized bits when the predicted residual becomes large, by switching the binarization method according to the magnitude of the predicted residual. The three-dimensional data encoding device may also add a threshold R_TH to the bitstream header or the like.

[0349] For example, when encoding is performed at a high bit rate, i.e., when the quantization scale is small, the quantization error is small and the prediction accuracy is high, which may result in a smaller prediction residual. Therefore, in this case, the three-dimensional data encoding device sets a large threshold R_TH. This reduces the likelihood of encoding binarized data with a threshold R_TH, improving encoding efficiency. Conversely, when encoding is performed at a low bit rate, i.e., when the quantization scale is large, the quantization error is large and the prediction accuracy is poor, which may result in a larger prediction residual. Therefore, in this case, the three-dimensional data encoding device sets a small threshold R_TH. This prevents a rapid increase in the bit length of the binarized data.

[0350] Furthermore, the 3D data encoding device may switch the threshold R_TH for each Level of Data (LoD) and add the LoD threshold R_TH to the header, etc. In other words, the 3D data encoding device may switch the binarization method for each LoD. For example, in the upper layers, the distance between 3D points is large, so the prediction accuracy may be poor and as a result the prediction residual may be large. Therefore, the 3D data encoding device can prevent a rapid increase in the bit length of the binarized data by setting a small threshold R_TH for the upper layers. Also, in the lower layers, the distance between 3D points is small, so the prediction accuracy may be high and as a result the prediction residual may be small. Therefore, the 3D data encoding device can improve encoding efficiency by setting a large threshold R_TH for each layer.

[0351] Figure 55 is a diagram showing an example of an exponential Golomb code, illustrating the relationship between the value before binarization (multi-level) and the bit after binarization (code). Note that the 0s and 1s shown in Figure 55 may also be inverted.

[0352] Furthermore, the three-dimensional data encoding device applies arithmetic coding to the binarized data of the prediction residuals. This improves coding efficiency. Note that when applying arithmetic coding, the probability trends of the occurrence of 0 and 1 for each bit may differ between the n-bit code (the n-bit binarized portion of the binarized data) and the remaining code (the portion binarized using exponential golombs). Therefore, the three-dimensional data encoding device may switch the method of applying arithmetic coding between the n-bit code and the remaining code.

[0353] For example, a three-dimensional data encoding device performs arithmetic encoding of an n-bit code using a different encoding table (probability table) for each bit. In this case, the three-dimensional data encoding device may change the number of encoding tables used for each bit. For example, the three-dimensional data encoding device uses one encoding table to perform arithmetic encoding of the first bit b0 of an n-bit code. The three-dimensional data encoding device then uses two encoding tables for the next bit b1. Furthermore, the three-dimensional data encoding device switches the encoding table used for arithmetic encoding of bit b1 depending on the value of b0 (0 or 1). Similarly, the three-dimensional data encoding device uses four encoding tables for the next bit b2. Furthermore, the three-dimensional data encoding device switches the encoding table used for arithmetic encoding of bit b2 depending on the values ​​of b0 and b1 (0 to 3).

[0354] Thus, when the three-dimensional data encoding device arithmetically encodes each bit bn-1 of an n-bit code, 2 n-1 The system uses a set of encoding tables. Furthermore, the three-dimensional data encoding device switches the encoding table used depending on the value (generation pattern) of the bits prior to bn-1. This allows the three-dimensional data encoding device to use the appropriate encoding table for each bit, thereby improving encoding efficiency.

[0355] Note that the three-dimensional data encoding device may reduce the number of encoding tables used for each bit. For example, when arithmetic-encoding each bit bn-1, the three-dimensional data encoding device may switch between two encoding tables according to the values (occurrence patterns) of the m bits (m < n - 1) before bn-1. This can improve the encoding efficiency while suppressing the number of encoding tables used for each bit. Note that the three-dimensional data encoding device may update the occurrence probabilities of 0 and 1 in each encoding table according to the values of the actually-occurred binary data. Also, the three-dimensional data encoding device may fix the occurrence probabilities of 0 and 1 in the encoding tables of some bits. This can suppress the number of updates of the occurrence probabilities, thus reducing the processing amount. m For example, when the n-bit code is b0b1b2…bn-1, the encoding table for b0 is one (CTb0). The encoding tables for b1 are two (CTb10, CTb11). Also, the encoding table to be used is switched according to the value (0 to 1) of b0. The encoding tables for b2 are four (CTb20, CTb21, CTb22, CTb23). Also, the encoding table to be used is switched according to the values (0 to 3) of b0 and b1. The encoding tables for bn-1 are two

[0356] (CTbn0, CTbn1, …, CTbn(2 n-1 -1)). Also, the encoding table to be used is switched according to the value (0 to 2 n-1 -1) of b0b1…bn-2. n-1

[0357] Note that the three-dimensional data encoding device may apply m-ary arithmetic encoding (m = 2 n ) that sets values from 0 to 2 n -1 without binary conversion to the n-bit code. Also, when the three-dimensional data encoding device performs arithmetic encoding of the n-bit code in m-ary, the three-dimensional data decoding device may also restore the n-bit code by m-ary arithmetic decoding.

[0358] Figure 56 is a diagram illustrating, for example, the processing when the remaining code is an exponential Golomb code. The remaining code, which is the part binarized using exponential Golomb, includes a prefix part and a suffix part, as shown in Figure 56. For example, a three-dimensional data encoding device switches the encoding table for the prefix part and the suffix part. That is, the three-dimensional data encoding device arithmetically encodes each bit in the prefix part using the encoding table for the prefix, and arithmetically encodes each bit in the suffix part using the encoding table for the suffix.

[0359] The three-dimensional data encoding device may update the probability of occurrence of 0 and 1 in each encoding table according to the actual values ​​of the binarized data that have occurred. Alternatively, the three-dimensional data encoding device may fix the probability of occurrence of 0 and 1 in either encoding table. This reduces the number of times the probability of occurrence is updated, thereby reducing the processing load. For example, the three-dimensional data encoding device may update the probability of occurrence for the prefix part and fix the probability of occurrence for the suffix part.

[0360] Furthermore, the three-dimensional data encoding device decodes the predicted residual after quantization by inverse quantization and reconstruction, and uses the decoded value, which is the decoded predicted residual, for predictions beyond the three-dimensional point to be encoded. Specifically, the three-dimensional data encoding device calculates the inverse quantized value by multiplying the predicted residual (quantized value) after quantization by the quantization scale, and obtains the decoded value (reconstructed value) by adding the inverse quantized value and the predicted value.

[0361] For example, the inverse quantization value a2iq of point a2 is calculated using the quantization value a2q of point a2 by (Equation A11). The inverse quantization value b2iq of point b2 is calculated using the quantization value b2q of point b2 by (Equation A12). Here, QS_LoD0 is the QS for LoD0, and QS_LoD1 is the QS for LoD1. In other words, the QS may be changed according to the LoD.

[0362] a2iq=a2q×QS_LoD0 (Formula A11) b2iq=b2q×QS_LoD1 (Formula A12)

[0363] For example, the decoded value a2rec of point a2 is calculated by adding the predicted value a2p of point a2 to the inverse quantized value a2iq of point a2, as shown in (Equation A13). The decoded value b2rec of point b2 is calculated by adding the predicted value b2p of point b2 to the inverse quantized value b2iq of point b2, as shown in (Equation A14).

[0364] a2rec=a2iq+a2p (formula A13) b2rec=b2iq+b2p (formula A14)

[0365] The following describes an example of bitstream syntax according to this embodiment. Figure 57 shows an example of attribute header syntax according to this embodiment. The attribute header is header information for attribute information. As shown in Figure 57, the attribute header includes hierarchy number information (NumLoD), three-dimensional point number information (NumOfPoint[i]), hierarchy threshold (Thres_Lod[i]), surrounding point number information (NumNeighorPoint[i]), prediction threshold (THd[i]), quantization scale (QS[i]), and binarization threshold (R_TH[i]).

[0366] The number of levels (NumLoD) indicates the number of levels of the Level of Data (LoD) used.

[0367] The three-dimensional point number information (NumOfPoint[i]) indicates the number of three-dimensional points belonging to hierarchy i. The three-dimensional data encoding device may also add total three-dimensional point number information (AllNumOfPoint), indicating the total number of three-dimensional points, to a separate header. In this case, the three-dimensional data encoding device does not need to add NumOfPoint[NumLoD-1], indicating the number of three-dimensional points belonging to the lowest layer, to the header. In this case, the three-dimensional data decoding device can calculate NumOfPoint[NumLoD-1] using (Equation A15). This reduces the amount of code in the header.

[0368]

Number

[0369] The hierarchical threshold (Thres_Lod[i]) is the threshold used for the setting of layer i. The three-dimensional data encoding device and the three-dimensional data decoding device configure LoDi such that the distance between each point in LoDi is greater than the threshold Thres_LoD[i]. Also, the three-dimensional data encoding device may not add the value of Thres_Lod[NumLoD - 1] (the bottom layer) to the header. In this case, the three-dimensional data decoding device estimates the value of Thres_Lod[NumLoD - 1] as 0. Thereby, the amount of code for the header can be reduced.

[0370] The surrounding point number information (NumNeighorPoint[i]) indicates the upper limit value of the number of surrounding points used for generating the predicted value of the three-dimensional points belonging to layer i. When the number M of surrounding points is less than NumNeighorPoint[i] (M < NumNeighorPoint[i]), the three-dimensional data encoding device may calculate the predicted value using M surrounding points. Also, when the three-dimensional data encoding device does not need to divide the value of NumNeighorPoint[i] for each LoD, it may add one piece of surrounding point number information (NumNeighorPoint) used for all LoDs to the header.

[0371] The prediction threshold (THd[i]) indicates the upper limit value of the distance between the surrounding three-dimensional points and the target three-dimensional point used for predicting the target three-dimensional point to be encoded or decoded at layer i. The three-dimensional data encoding device and the three-dimensional data decoding device do not use, for prediction, three-dimensional points whose distance from the target three-dimensional point is farther than THd[i]. Note that when the three-dimensional data encoding device does not need to divide the value of THd[i] for each LoD, it may add one prediction threshold (THd) used for all LoDs to the header.

[0372] The quantization scale (QS[i]) indicates the quantization scale used for quantization and inverse quantization of layer i.

[0373] The binarization threshold (R_TH[i]) is a threshold used to switch the binarization method for the predicted residuals of three-dimensional points belonging to hierarchy i. For example, if the predicted residual is less than the threshold R_TH, the three-dimensional data encoder binarizes the predicted residual pu with a fixed number of bits. If the predicted residual is greater than or equal to the threshold R_TH, it binarizes the binarized data of threshold R_TH and the value of (pu - R_TH) using exponential golomb. If it is not necessary to switch the value of R_TH[i] for each LoD, the three-dimensional data encoder may add a single binarization threshold (R_TH) used for all LoDs to the header.

[0374] Note that R_TH[i] may be the maximum value that can be represented by n bits. For example, R_TH is 63 for 6 bits and 255 for 8 bits. Alternatively, instead of encoding the maximum value that can be represented by n bits as the binarization threshold, the 3D data encoder may encode the number of bits. For example, the 3D data encoder may add the value 6 to the header when R_TH[i]=63, and the value 8 when R_TH[i]=255. Alternatively, the 3D data encoder may define the minimum number of bits that represent R_TH[i] (minimum number of bits) and add the relative number of bits from the minimum value to the header. For example, the 3D data encoder may add the value 0 to the header when R_TH[i]=63 and the minimum number of bits is 6, and add the value 2 to the header when R_TH[i]=255 and the minimum number of bits is 6.

[0375] Furthermore, the three-dimensional data encoding device may entropy encode at least one of NumLoD, Thres_Lod[i], NumNeighborPoint[i], THd[i], QS[i], and R_TH[i] and add it to the header. For example, the three-dimensional data encoding device may binarize each value and then arithmetic encode it. Alternatively, the three-dimensional data encoding device may encode each value with a fixed length to reduce processing load.

[0376] Furthermore, the three-dimensional data encoding device does not need to include at least one of NumLoD, Thres_Lod[i], NumNeighborPoint[i], THd[i], QS[i], and R_TH[i] in the header. For example, at least one of these values ​​may be defined in the profile or level of a standard or similar specification. This can reduce the number of bits in the header.

[0377] Figure 58 shows an example of the syntax of attribute data according to this embodiment. This attribute data includes encoded data of attribute information for multiple three-dimensional points. As shown in Figure 58, the attribute data includes an n-bit code and a remaining code.

[0378] An n-bit code is the encoded data or a portion thereof of the predicted residual of the attribute information value. The bit length of the n-bit code depends on the value of R_TH[i]. For example, if the value of R_TH[i] is 63, the n-bit code is 6 bits, and if the value of R_TH[i] is 255, the n-bit code is 8 bits.

[0379] The remaining code is the coded data of the predicted residual of the attribute information value, coded using exponential golomb. This remaining code is coded or coded when the n-bit code is the same as R_TH[i]. The three-dimensional data decoder decodes the predicted residual by adding the value of the n-bit code and the value of the remaining code. If the n-bit code is not the same as R_TH[i], the remaining code does not need to be coded or coded.

[0380] The following describes the processing flow in the three-dimensional data encoding device. Figure 59 is a flowchart of the three-dimensional data encoding process performed by the three-dimensional data encoding device.

[0381] First, the three-dimensional data encoding device encodes the positional information (geometry) (S3001). For example, three-dimensional data encoding is performed using an octave tree representation.

[0382] The three-dimensional data encoding device reassigns the attribute information of the original three-dimensional point to the changed three-dimensional point if the position of the three-dimensional point changes due to quantization or the like after encoding the position information (S3002). For example, the three-dimensional data encoding device performs the reassignment by interpolating the value of the attribute information according to the amount of change in position. For example, the three-dimensional data encoding device detects N three-dimensional points that are close to the changed three-dimensional position and performs a weighted average of the attribute information values ​​of the N three-dimensional points. For example, in the weighted average, the three-dimensional data encoding device determines the weights based on the distance from the changed three-dimensional position to each of the N three-dimensional points. Then, the three-dimensional data encoding device determines the value obtained by the weighted average as the attribute information value of the changed three-dimensional point. Furthermore, if two or more three-dimensional points change to the same three-dimensional position due to quantization or the like, the three-dimensional data encoding device may assign the average value of the attribute information of the two or more three-dimensional points before the change as the attribute information value of the changed three-dimensional point.

[0383] Next, the three-dimensional data encoding device encodes the reassigned attribute information (Attribute) (S3003). For example, if the three-dimensional data encoding device encodes multiple types of attribute information, it may encode the multiple types of attribute information sequentially. For example, if the three-dimensional data encoding device encodes color and reflectance as attribute information, it may generate a bitstream in which the encoded result of reflectance is appended after the encoded result of color. Note that the order of the multiple encoded results of attribute information appended to the bitstream is not limited to this order and may be any order.

[0384] Furthermore, the three-dimensional data encoding device may add information to the header or elsewhere indicating the starting location of the encoded data for each attribute information within the bitstream. This allows the three-dimensional data decoding device to selectively decode the attribute information that needs to be decoded, thus omitting the decoding process for attribute information that does not need to be decoded. Therefore, the processing load of the three-dimensional data decoding device can be reduced. In addition, the three-dimensional data encoding device may encode multiple types of attribute information in parallel and integrate the encoding results into a single bitstream. This allows the three-dimensional data encoding device to encode multiple types of attribute information at high speed.

[0385] Figure 60 is a flowchart of the attribute information encoding process (S3003). First, the three-dimensional data encoding device sets the Level of Data (LoD) (S3011). In other words, the three-dimensional data encoding device assigns each three-dimensional point to one of several LoDs.

[0386] Next, the three-dimensional data encoding device starts a loop for each Level of Data (LoD) (S3012). In other words, the three-dimensional data encoding device repeatedly performs the processes in steps S3013 to S3021 for each LoD.

[0387] Next, the three-dimensional data encoding device starts a loop for each three-dimensional point (S3013). In other words, the three-dimensional data encoding device repeats the process from steps S3014 to S3020 for each three-dimensional point.

[0388] First, the three-dimensional data encoding device searches for multiple surrounding points, which are three-dimensional points that exist around the target three-dimensional point to be processed, in order to calculate the predicted value of the target three-dimensional point (S3014). Next, the three-dimensional data encoding device calculates the weighted average of the attribute information values ​​of the multiple surrounding points and sets the obtained value as the predicted value P (S3015). Next, the three-dimensional data encoding device calculates the prediction residual, which is the difference between the attribute information of the target three-dimensional point and the predicted value (S3016). Next, the three-dimensional data encoding device calculates the quantized value by quantizing the prediction residual (S3017). Next, the three-dimensional data encoding device arithmetically encodes the quantized value (S3018).

[0389] Furthermore, the three-dimensional data encoding device calculates the inverse quantized value by inverse quantizing the quantized value (S3019). Next, the three-dimensional data encoding device generates the decoded value by adding the predicted value to the inverse quantized value (S3020). Next, the three-dimensional data encoding device terminates the loop for three-dimensional points (S3021). Furthermore, the three-dimensional data encoding device terminates the loop for LoD (Line of Data) (S3022).

[0390] The following describes the three-dimensional data decoding process in a three-dimensional data decoding device that decodes the bitstream generated by the three-dimensional data encoding device described above.

[0391] The three-dimensional data decoder generates decoded binarized data by arithmetic decoding the binarized attribute information data within the bitstream generated by the three-dimensional data encoder in the same manner as the three-dimensional data encoder. If the three-dimensional data encoder switches the application method of arithmetic coding between the n-bit binarized portion (n-bit code) and the exponential golomb binarized portion (remaining code), the three-dimensional data decoder will perform decoding accordingly when applying arithmetic decoding.

[0392] For example, a three-dimensional data decoder performs arithmetic decoding of an n-bit code using a different coding table (decoding table) for each bit. In this case, the three-dimensional data decoder may change the number of coding tables used for each bit. For example, the first bit b0 of an n-bit code is decoded using one coding table. The three-dimensional data decoder then uses two coding tables for the next bit b1. Furthermore, the three-dimensional data decoder switches the coding table used for arithmetic decoding of bit b1 depending on the value of b0 (0 or 1). Similarly, the three-dimensional data decoder uses four coding tables for the next bit b2. Furthermore, the three-dimensional data decoder switches the coding table used for arithmetic decoding of bit b2 depending on the values ​​of b0 and b1 (0 to 3).

[0393] Thus, when the three-dimensional data decoder arithmetically decodes each bit bn-1 of the n-bit code, it uses 2 n-1 encoding tables. Also, the three-dimensional data decoder switches the encoding table to be used according to the value (generation pattern) of the bits before bn-1. Thereby, the three-dimensional data decoder can appropriately decode a bit stream with improved encoding efficiency by using an appropriate encoding table for each bit.

[0394] Note that the three-dimensional data decoder may reduce the number of encoding tables used for each bit. For example, when the three-dimensional data decoder arithmetically decodes each bit bn-1, it may switch between 2 m encoding tables according to the value (generation pattern) of m bits (m < n-1) before bn-1. Thereby, the three-dimensional data decoder can appropriately decode a bit stream with improved encoding efficiency while suppressing the number of encoding tables used for each bit. Note that the three-dimensional data decoder may update the occurrence probabilities of 0 and 1 in each encoding table according to the value of the actually generated binarized data. Also, the three-dimensional data decoder may fix the occurrence probabilities of 0 and 1 in the encoding tables for some bits. Thereby, the number of updates of the occurrence probability can be suppressed, and the processing amount can be reduced.

[0395] For example, when the n-bit code is b0b1b2…bn-1, the encoding table for b0 is 1 (CTb0). The encoding table for b1 is 2 (CTb10, CTb11). Also, the encoding table is switched according to the value (0 to 1) of b0. The encoding table for b2 is 4 (CTb20, CTb21, CTb22, CTb23). Also, the encoding table is switched according to the values (0 to 3) of b0 and b1. The encoding table for bn-1 is 2 n-1 (CTbn0, CTbn1, …, CTbn(2 n-1 -1)). Also, the encoding table is switched according to the value (0 to 2 n-1 -1) of b0b1…bn-2.

[0396] Figure 61 illustrates, for example, the processing when the remaining code is an exponential Golomb code. The portion (remaining code) that the three-dimensional data encoding device has binarized and encoded using exponential Golomb includes a prefix section and a suffix section, as shown in Figure 61. For example, the three-dimensional data decoding device switches the encoding table between the prefix section and the suffix section. That is, the three-dimensional data decoding device arithmetically decodes each bit in the prefix section using the encoding table for the prefix, and arithmetically decodes each bit in the suffix section using the encoding table for the suffix.

[0397] The three-dimensional data decoder may update the probability of occurrence of 0 and 1 in each encoding table according to the value of the binarized data generated during decoding. Alternatively, the three-dimensional data decoder may fix the probability of occurrence of 0 and 1 in either encoding table. This reduces the number of updates to the occurrence probability, thereby reducing the processing load. For example, the three-dimensional data decoder may update the occurrence probability for the prefix section and fix the occurrence probability for the suffix section.

[0398] Furthermore, the three-dimensional data decoder decodes the quantized prediction residuals (unsigned integer values) by multi-leveling the binarized data of the arithmetic-decoded prediction residuals according to the encoding method used by the three-dimensional data encoding device. The three-dimensional data decoder first calculates the value of the decoded n-bit code by arithmetic decoding the binarized data of the n-bit code. Next, the three-dimensional data decoder compares the value of the n-bit code with the value of R_TH.

[0399] The three-dimensional data decoder determines that if the value of the n-bit code matches the value of R_TH, the next bit encoded with exponential golom exists, and arithmetic decoding is performed on the remaining code, which is the binarized data encoded with exponential golom. The three-dimensional data decoder then calculates the value of the remaining code from the decoded remaining code using a reverse lookup table that shows the relationship between the remaining code and its value. Figure 62 is a diagram showing an example of a reverse lookup table that shows the relationship between the remaining code and its value. Next, the three-dimensional data decoder obtains the multi-level quantized predicted residual by adding the obtained value of the remaining code to R_TH.

[0400] On the other hand, if the value of the n-bit code and the value of R_TH do not match (the value is smaller than R_TH), the three-dimensional data decoder uses the value of the n-bit code as the predicted residual after multi-level quantization. This allows the three-dimensional data decoder to appropriately decode the bitstream generated by the three-dimensional data encoding device by switching the binarization method according to the value of the predicted residual.

[0401] Furthermore, if the threshold R_TH is attached to the bitstream header, the three-dimensional data decoder may decode the value of the threshold R_TH from the header and switch the decoding method using the decoded value of the threshold R_TH. Also, if the threshold R_TH is attached to the header for each Level of Data (LoD), the three-dimensional data decoder may switch the decoding method using the decoded threshold R_TH for each LoD.

[0402] For example, if the threshold R_TH is 63 and the value of the decoded n-bit code is 63, the three-dimensional data decoder obtains the value of the remaining code by decoding the remaining code using exponential golomb. For example, in the example shown in Figure 62, the remaining code is 00100, and the value of the remaining code is obtained as 3. Next, the three-dimensional data decoder obtains the predicted residual value 66 by adding the threshold R_TH value 63 and the remaining code value 3.

[0403] Furthermore, if the value of the decoded n-bit code is 32, the three-dimensional data decoder sets the value of the n-bit code (32) to the value of the predicted residual.

[0404] Furthermore, the three-dimensional data decoder converts the decoded quantized prediction residual from an unsigned integer value to a signed integer value, for example, by the reverse of the processing performed in the three-dimensional data encoding device. This allows the three-dimensional data decoder to properly decode bitstreams generated without considering the occurrence of negative integers when entropy coding the prediction residual. Note that the three-dimensional data decoder does not necessarily need to convert unsigned integer values ​​to signed integer values; for example, when decoding a bitstream generated by separately entropy coding the sign bit, the sign bit may be decoded.

[0405] The three-dimensional data decoder generates decoded values ​​by decoding the predicted residuals after quantization, which have been converted to signed integer values, through inverse quantization and reconstruction. The three-dimensional data decoder also uses the generated decoded values ​​to predict the three-dimensional points and beyond that of the target of decoding. Specifically, the three-dimensional data decoder calculates the inverse quantized value by multiplying the predicted residuals after quantization by the decoded quantization scale, and then obtains the decoded value by adding the inverse quantized value and the predicted value.

[0406] The decoded unsigned integer value (unsigned quantized value) is converted to a signed integer value by the following process: If the LSB (least significant bit) of the decoded unsigned integer value a2u is 1, the three-dimensional data decoder sets the signed integer value a2q to -((a2u+1)>>1). If the LSB of the unsigned integer value a2u is not 1, the three-dimensional data decoder sets the signed integer value a2q to (a2u>>1).

[0407] Similarly, the 3D data decoder sets the signed integer b2q to -((b2u+1)>>1) if the LSB of the decoded unsigned integer b2u is 1. The 3D data decoder sets the signed integer b2q to (b2u>>1) if the LSB of the unsigned integer n2u is not 1.

[0408] Furthermore, the details of the inverse quantization and reconstruction process using the three-dimensional data decoding device are the same as those of the inverse quantization and reconstruction process using the three-dimensional data encoding device.

[0409] The following describes the processing flow in the three-dimensional data decoding device. Figure 63 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding device. First, the three-dimensional data decoding device decodes the position information (geometry) from the bitstream (S3031). For example, the three-dimensional data decoding device performs decoding using an octave tree representation.

[0410] Next, the three-dimensional data decoder decodes attribute information from the bitstream (S3032). For example, if the three-dimensional data decoder decodes multiple types of attribute information, it may decode the multiple types of attribute information in order. For example, if the three-dimensional data decoder decodes color and reflectance as attribute information, it decodes the color encoding result and the reflectance encoding result in the order in which they are added to the bitstream. For example, if the reflectance encoding result is added after the color encoding result in the bitstream, the three-dimensional data decoder decodes the color encoding result, and then decodes the reflectance encoding result. The three-dimensional data decoder may decode the encoding results of the attribute information added to the bitstream in any order.

[0411] Furthermore, the three-dimensional data decoding device may obtain information indicating the start location of the encoded data for each attribute information within the bitstream by decoding the header, etc. This allows the three-dimensional data decoding device to selectively decode the attribute information that needs to be decoded, thus omitting the decoding process for attribute information that does not need to be decoded. Therefore, the processing load of the three-dimensional data decoding device can be reduced. In addition, the three-dimensional data decoding device may decode multiple types of attribute information in parallel and integrate the decoding results into a single three-dimensional point cloud. This allows the three-dimensional data decoding device to decode multiple types of attribute information at high speed.

[0412] Figure 64 is a flowchart of the attribute information decoding process (S3032). First, the three-dimensional data decoding device sets the Level of Direction (LoD) (S3041). That is, the three-dimensional data decoding device assigns each of the multiple three-dimensional points having decoded position information to one of the multiple LoDs. For example, this assignment method is the same as the assignment method used in the three-dimensional data encoding device.

[0413] Next, the three-dimensional data decoding device starts a loop for each Level of Data (LoD) (S3042). In other words, the three-dimensional data decoding device repeats the process from steps S3043 to S3049 for each LoD.

[0414] Next, the three-dimensional data decoding device starts a loop for each three-dimensional point (S3043). In other words, the three-dimensional data decoding device repeats the process from steps S3044 to S3048 for each three-dimensional point.

[0415] First, the three-dimensional data decoding device searches for multiple surrounding points, which are three-dimensional points that exist around the target three-dimensional point to be processed, in order to calculate the predicted value of the target three-dimensional point (S3044). Next, the three-dimensional data decoding device calculates the weighted average of the attribute information values ​​of the multiple surrounding points and sets the obtained value as the predicted value P (S3045). These processes are the same as those performed in the three-dimensional data encoding device.

[0416] Next, the three-dimensional data decoding device arithmetically decodes the quantized values ​​from the bitstream (S3046). The three-dimensional data decoding device also calculates the inverse quantized values ​​by inverse quantizing the decoded quantized values ​​(S3047). Next, the three-dimensional data decoding device generates the decoded values ​​by adding the predicted values ​​to the inverse quantized values ​​(S3048). Next, the three-dimensional data decoding device terminates the loop for each three-dimensional point (S3049). The three-dimensional data decoding device also terminates the loop for each Line of Data (LoD) (S3050).

[0417] Next, the configuration of the three-dimensional data encoding device and the three-dimensional data decoding device according to this embodiment will be described. Figure 65 is a block diagram showing the configuration of the three-dimensional data encoding device 3000 according to this embodiment. This three-dimensional data encoding device 3000 includes a position information encoding unit 3001, an attribute information reassignment unit 3002, and an attribute information encoding unit 3003.

[0418] The attribute information encoding unit 3003 encodes the position information (geometry) of multiple three-dimensional points included in the input point cloud. The attribute information reassignment unit 3002 reassigns the attribute information values ​​of multiple three-dimensional points included in the input point cloud using the encoding and decoding results of the position information. The attribute information encoding unit 3003 encodes the reassigned attribute information. The three-dimensional data encoding device 3000 generates a bitstream containing the encoded position information and encoded attribute information.

[0419] Figure 66 is a block diagram showing the configuration of a three-dimensional data decoding device 3010 according to this embodiment. This three-dimensional data decoding device 3010 includes a position information decoding unit 3011 and an attribute information decoding unit 3012.

[0420] The position information decoding unit 3011 decodes the position information (geometry) of multiple three-dimensional points from the bitstream. The attribute information decoding unit 3012 decodes the attribute information (attribute) of multiple three-dimensional points from the bitstream. The three-dimensional data decoding device 3010 generates an output point group by combining the decoded position information and the decoded attribute information.

[0421] (Embodiment 5) In the following, the three-dimensional point to be encoded may be referred to as the first three-dimensional point, and the three-dimensional points surrounding it may be referred to as the second three-dimensional point.

[0422] For example, in generating predicted values ​​for the attribute information of a three-dimensional point, the attribute value of the closest three-dimensional point among the encoded and decoded surrounding three-dimensional points of the three-dimensional point to be encoded may be used directly as the predicted value. Alternatively, prediction mode information (PredMode) may be added to each three-dimensional point during prediction value generation, allowing the generation of a predicted value by selecting one from multiple predicted values. In other words, for example, in a total of M prediction modes, the average value may be assigned to prediction mode 0, the attribute value of three-dimensional point A to prediction mode 1, ..., the attribute value of three-dimensional point Z to prediction mode M-1, and the prediction mode used for prediction may be added to the bitstream for each three-dimensional point. Thus, the first prediction mode value, which indicates the first prediction mode in which the average of the attribute information of the surrounding three-dimensional points is calculated as the predicted value, may be smaller than the second prediction mode value, which indicates the second prediction mode in which the attribute information of the surrounding three-dimensional points itself is calculated as the predicted value. Here, the "average value" calculated in prediction mode 0 is the average of the attribute values ​​of the three-dimensional points surrounding the three-dimensional point to be encoded.

[0423] Figure 67 is a diagram showing a first example of a table showing the predicted values ​​calculated in each prediction mode according to Embodiment 5. Figure 68 is a diagram showing an example of attribute information used for the predicted values ​​according to Embodiment 5. Figure 69 is a diagram showing a second example of a table showing the predicted values ​​calculated in each prediction mode according to Embodiment 5.

[0424] The number of prediction modes M may be appended to the bitstream. Alternatively, the number of prediction modes M may be defined by the standard's profile, level, etc., without being appended to the bitstream. Furthermore, the number of prediction modes M may be calculated from the number of three-dimensional points N used for prediction. For example, the number of prediction modes M may be calculated as M = N + 1.

[0425] Note that the table shown in Figure 67 is an example for the case where the number of three-dimensional points used for prediction is N=4 and the number of prediction modes is M=5. The predicted value of the attribute information of point b2 can be generated using the attribute information of points a0, a1, a2, and b1. When selecting one prediction mode from multiple prediction modes, one may select a prediction mode that generates the attribute values ​​of points a0, a1, a2, and b1 as predicted values ​​based on the distance information from point b2 to each of points a0, a1, a2, and b1. A prediction mode is assigned to each three-dimensional point to be encoded. The predicted value is calculated according to the value corresponding to the assigned prediction mode.

[0426] The table shown in Figure 69 is an example, similar to Figure 67, where the number of three-dimensional points used for prediction is N=4 and the number of prediction modes is M=5. The predicted value of the attribute information of point a2 can be generated using the attribute information of points a0 and a1. When selecting one prediction mode from multiple prediction modes, one may select a prediction mode that generates the attribute values ​​of points a0 and a1 as predicted values ​​based on the distance information from point a2 to each of points a0 and a1. A prediction mode is assigned to each three-dimensional point to be encoded. The predicted value is calculated according to the value corresponding to the assigned prediction mode.

[0427] Furthermore, if the number of adjacent points, i.e., the number of surrounding three-dimensional points N, is less than four, as in the case of point a2 above, the prediction mode for which no prediction value has been assigned may be marked as "not available" in the table.

[0428] Note that the assignment of prediction mode values ​​may be determined in order of distance from the three-dimensional point to be encoded. For example, the prediction mode value indicating multiple prediction modes is smaller the closer the distance from the three-dimensional point to be encoded to the surrounding three-dimensional points that have attribute information to be used as prediction values. In the example in Figure 67, points b1, a2, a1, and a0 are shown to be in the order of proximity to point b2, which is the three-dimensional point to be encoded. For example, in calculating the prediction value, the attribute information of point b1 is calculated as a prediction value in the prediction mode indicated by the prediction mode value "1" among the two or more prediction modes, and the attribute information of point a2 is calculated as a prediction value in the prediction mode indicated by the prediction mode value "2". In this way, the prediction mode value indicating the prediction mode in which the attribute information of point b1 is calculated as a prediction value is smaller than the prediction mode value indicating the prediction mode in which the attribute information of point a2, which is located further away from point b2 than point b1, is calculated as a prediction value.

[0429] This allows for assigning smaller prediction mode values ​​to points that are closer in distance and therefore more likely to be selected, thereby reducing the number of bits required to encode the prediction mode values. Alternatively, smaller prediction mode values ​​may be preferentially assigned to three-dimensional points belonging to the same Level of Deposition (LoD) as the three-dimensional point being encoded.

[0430] Figure 70 shows a third example of a table showing the predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the third example is an example where the attribute information used for the predicted value is a value based on the color information (YUV) of the surrounding three-dimensional points. Thus, the attribute information used for the predicted value may be color information indicating the color of the three-dimensional points.

[0431] As shown in Figure 70, the predicted value calculated in the prediction mode indicated by a prediction mode value of "0" is the average of each YUV component that defines the YUV color space. Specifically, this predicted value includes a weighted average Yave of Yb1, Ya2, Ya1, and Ya0, which are the Y component values ​​corresponding to points b1, a2, a1, and a0 respectively; a weighted average Uave of Ub1, Ua2, Ua1, and Ua0, which are the U component values ​​corresponding to points b1, a2, a1, and a0 respectively; and a weighted average Vave of Vb1, Va2, Va1, and Va0, which are the V component values ​​corresponding to points b1, a2, a1, and a0 respectively. Furthermore, the predicted values ​​calculated in the prediction modes indicated by prediction mode values ​​of "1" to "4" each include the color information of the surrounding three-dimensional points b1, a2, a1, and a0, respectively. The color information is represented by a combination of the Y, U, and V component values.

[0432] In Figure 70, the color information is shown using values ​​defined in the YUV color space, but it may also be shown using values ​​defined in the RGB color space or other color spaces.

[0433] Thus, in calculating the predicted value, two or more averages or attribute pieces of information may be calculated as the predicted value of the prediction mode. Furthermore, each of the two or more averages or attribute pieces of information may represent the values ​​of two or more components that define the color space.

[0434] For example, if a prediction mode is selected in the table in Figure 70, where the prediction mode value is "2", the Y, U, and V components of the attribute values ​​of the three-dimensional point to be encoded may be used as the prediction values ​​Ya2, Ua2, and Va2, respectively, for encoding. In this case, the prediction mode value "2" is added to the bitstream.

[0435] Figure 71 shows a fourth example of a table showing the predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the fourth example is an example where the attribute information used for the predicted value is a value based on the reflectance information of the surrounding three-dimensional points. The reflectance information is, for example, information indicating the reflectance R.

[0436] As shown in Figure 71, the predicted values ​​calculated in the prediction mode indicated by a prediction mode value of "0" are the weighted average Rave of the reflectances Rb1, Ra2, Ra1, and Ra0 corresponding to points b1, a2, a1, and a0, respectively. The predicted values ​​calculated in the prediction modes indicated by prediction mode values ​​of "1" to "4" are the reflectances Rb1, Ra2, Ra1, and Ra0 of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0437] For example, if a prediction mode is selected in the table in Figure 71, where the prediction mode value is "3", the reflectance of the attribute value of the three-dimensional point to be encoded may be used as the prediction value Ra1 for encoding. In this case, "3" as the prediction mode value is added to the bitstream.

[0438] As shown in Figures 70 and 71, attribute information may include first attribute information and second attribute information of a different type from the first attribute information. The first attribute information may be, for example, color information. The second attribute information may be, for example, reflectance information. In calculating the predicted value, the first predicted value may be calculated using the first attribute information, and the second predicted value may be calculated using the second attribute information.

[0439] When attribute information has multiple components, such as color information in the YUV color space or RGB color space, the predicted values ​​may be calculated using separate prediction modes for each component. For example, in the YUV space, the predicted values ​​using the Y, U, and V components may be calculated using the prediction mode selected for each component. For instance, a prediction mode value may be selected for each of the following: prediction mode Y for calculating the predicted value using the Y component, prediction mode U for calculating the predicted value using the U component, and prediction mode V for calculating the predicted value using the V component. In this case, the prediction mode values ​​indicating the prediction mode for each component may be the values ​​from the tables in Figures 72 to 74 described later, and these prediction mode values ​​may be added to the bitstream. Although the above explanation focused on the YUV color space, the same principles apply to the RGB color space.

[0440] Furthermore, predicted values ​​containing two or more components of the attribute information may be calculated using a common prediction mode. For example, in the case of the YUV color space, prediction mode Y may be selected for calculating predicted values ​​using the Y component, and prediction mode UV may be selected for calculating predicted values ​​using the UV component. In this case, the prediction mode values ​​indicating the prediction mode of each component may be the values ​​from the tables in Figures 72 and 75 described later, and these prediction mode values ​​may be added to the bitstream.

[0441] Figure 72 shows a fifth example of a table showing the predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the fifth example is an example in which the attribute information used for the predicted value is the Y component value of the color information of the surrounding three-dimensional points.

[0442] As shown in Figure 72, the predicted value calculated in prediction mode Y, where the prediction mode value is "0", is the weighted average Yave of the Y component values ​​Yb1, Ya2, Ya1, and Ya0 corresponding to points b1, a2, a1, and a0, respectively. The predicted values ​​calculated in prediction modes where the prediction mode value is "1" to "4" are the Y component values ​​Yb1, Ya2, Ya1, and Ya0 of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0443] For example, if prediction mode Y, indicated by the prediction mode value "2" in the table in Figure 72, is selected, the Y component of the attribute value of the three-dimensional point to be encoded may be used as the prediction value Ya2 for encoding. In this case, the prediction mode value "2" is added to the bitstream.

[0444] Figure 73 shows a sixth example of a table showing the predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the sixth example is an example in which the attribute information used for the predicted value is the value of the U component of the color information of the surrounding three-dimensional points.

[0445] As shown in Figure 73, the predicted value calculated in prediction mode U, where the prediction mode value is "0", is the weighted average Uave of Ub1, Ua2, Ua1, and Ua0, which are the U component values ​​corresponding to points b1, a2, a1, and a0, respectively. The predicted values ​​calculated in prediction modes where the prediction mode value is "1" to "4" are the U component values ​​Ub1, Ua2, Ua1, and Ua0 of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0446] For example, if prediction mode U, indicated by a prediction mode value of "1" in the table in Figure 73, is selected, the U component of the attribute value of the three-dimensional point to be encoded may be used as the prediction value Ub1 for encoding. In this case, "1" as the prediction mode value is added to the bitstream.

[0447] Figure 74 shows a seventh example of a table showing the predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the seventh example is an example where the attribute information used for the predicted value is the value of the V component of the color information of the surrounding three-dimensional points.

[0448] As shown in Figure 74, the predicted value calculated in prediction mode V, where the prediction mode value is "0", is the weighted average Vave of the V component values ​​Vb1, Va2, Va1, and Va0 corresponding to points b1, a2, a1, and a0, respectively. The predicted values ​​calculated in prediction modes where the prediction mode value is "1" to "4" are the V component values ​​Vb1, Va2, Va1, and Va0 of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0449] For example, if prediction mode V, indicated by the prediction mode value "4" in the table in Figure 74, is selected, the V component of the attribute value of the three-dimensional point to be encoded may be used as the prediction value Va0 for encoding. In this case, the prediction mode value "4" is added to the bitstream.

[0450] Figure 75 shows an eighth example of a table showing the predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the eighth example is an example where the attribute information used for the predicted value is the value of the U component and the V component of the color information of the surrounding three-dimensional points.

[0451] As shown in Figure 75, the predicted values ​​calculated in prediction mode U, where the prediction mode value is "0", include the weighted average Uave of Ub1, Ub2, Ua1, and Ua0, which are the U component values ​​corresponding to points b1, a2, a1, and a0, respectively, and the weighted average Vave of Vb1, Vb2, Va1, and Va0, which are the V component values ​​corresponding to points b1, a2, a1, and a0, respectively. In addition, the predicted values ​​calculated in prediction modes where the prediction mode value is "1" to "4" include the U component values ​​and V component values ​​of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0452] For example, if prediction mode UV, indicated by a prediction mode value of "1" in the table in Figure 75, is selected, the U and V components of the attribute values ​​of the three-dimensional point to be encoded may be used as prediction values ​​Ub1 and Vb1, respectively, for encoding. In this case, "1" as the prediction mode value is added to the bitstream.

[0453] The prediction mode during encoding may be selected by RD optimization. For example, one could calculate the cost(P) for selecting a certain prediction mode P and select the prediction mode P that minimizes cost(P). The cost(P) may be calculated using equation D1, for example, with the prediction residual(P) when using the predicted value of prediction mode P, the number of bits(P) required to encode prediction mode P, and the tuning parameter λ value.

[0454] cost(P)=abs(residual(P))+λ×bit(P)...(Formula D1)

[0455] abs(x) represents the absolute value of x. You can also use the square of x instead of abs(x).

[0456] By using the above equation D1, it becomes possible to select a prediction mode that balances the magnitude of the prediction residual with the number of bits required to encode the prediction mode. Note that the adjustment parameter λ may be set to different values ​​depending on the quantization scale. For example, when the quantization scale is small (high bitrate), a smaller λ value can be used to select a prediction mode that minimizes the prediction residual (P) and improve prediction accuracy as much as possible. When the quantization scale is large (low bitrate), a larger λ value can be used to select an appropriate prediction mode while considering the number of bits (bit(P)) required to encode the prediction mode P.

[0457] A small quantization scale is, for example, smaller than the first quantization scale. A large quantization scale is, for example, larger than the second quantization scale which is greater than or equal to the first quantization scale. Furthermore, the smaller the quantization scale, the smaller the λ value may be set to.

[0458] The predicted residual(P) is calculated by subtracting the predicted value of prediction mode P from the attribute value of the three-dimensional point to be coded. Alternatively, instead of using the predicted residual(P) when calculating the cost, the predicted residual(P) can be quantized and dequantized, added to the predicted value to obtain the decoded value, and the difference (coding error) between the original three-dimensional point attribute value and the decoded value using prediction mode P can be reflected in the cost value. This makes it possible to select a prediction mode with a small coding error.

[0459] The number of bits bit(P) required to encode the prediction mode P can be the number of bits after binarization, for example, if the prediction mode is binarized before encoding. For example, if the number of prediction modes M=5, the prediction mode value indicating the prediction mode may be binarized using a truncated unary code with a maximum value of 5, as shown in Figure 76. In this case, 1 bit is used when the prediction mode value is "0", 2 bits when the prediction mode value is "1", 3 bits when the prediction mode value is "2", and 4 bits when the prediction mode value is "4" or "5", as the number of bits bit(P) required to encode each prediction mode value. By using a truncated unary code, the number of bits decreases as the value of the prediction mode becomes smaller. Therefore, the amount of code in the prediction mode value indicating the prediction mode can be reduced, as it can calculate a prediction value that is likely to be selected, such as the average value calculated as the prediction value when the prediction mode value is "0", or the attribute information of the three-dimensional point calculated as the prediction value when the prediction mode value is "1", that is, the attribute information of the three-dimensional point that is close in distance to the three-dimensional point to be encoded, for example, a prediction value that is likely to have the minimum cost(P).

[0460] Furthermore, if the maximum number of prediction modes is not determined, the prediction mode values ​​indicating the prediction modes may be binarized using a unary code, as shown in Figure 77. Also, if the probability of occurrence of each prediction mode is close, the prediction mode values ​​indicating the prediction modes may be binarized using a fixed code to reduce the code size, as shown in Figure 78.

[0461] Furthermore, the number of bits (bit(P)) required to encode the prediction mode value indicating prediction mode P can be calculated by arithmetic coding the binarized data of the prediction mode value indicating prediction mode P, and the resulting code value (bit(P)) can be used as the value of bit(P). This allows for the calculation of the cost using a more accurate number of bits (bit(P)), making it possible to select a more appropriate prediction mode.

[0462] Figure 76 shows a first example of a binarization table when predictive mode values ​​are binarized and encoded according to Embodiment 5. Specifically, the first example is one in which the predictive mode values ​​are binarized using a truncated unary code when the number of predictive modes M=5.

[0463] Figure 77 shows a second example of a binarization table when predictive mode values ​​are binarized and encoded according to Embodiment 5. Specifically, the second example is an example in which predictive mode values ​​are binarized using unary code when the number of predictive modes M=5.

[0464] Figure 78 shows a third example of a binarization table when predictive mode values ​​are binarized and encoded according to Embodiment 5. Specifically, the third example is an example in which predictive mode values ​​are binarized using a fixed code when the number of predictive modes M=5.

[0465] The PredMode value, which indicates the PredMode, may be arithmetic encoded after binarization and appended to the bitstream. As described above, the PredMode value may be binarized using, for example, a truncated unary code with the number of PredModes M. In this case, the maximum number of bits after binarization of the PredMode value is M-1.

[0466] Furthermore, the binary data after binarization may be arithmetic encoded using an encoding table. In this case, encoding efficiency may be improved by, for example, switching the encoding table for each bit of the binary data. Alternatively, to reduce the number of encoding tables, the first bit of the binary data may be encoded using encoding table A for one bit, and each of the remaining bits (remaining bits) may be encoded using encoding table B for remaining bits. For example, when encoding the binarized data "1110" with a predicted mode value of "3" shown in Figure 79, the first bit "1" may be encoded using encoding table A, and each of the remaining bits "110" may be encoded using encoding table B.

[0467] Figure 79 is a diagram illustrating an example of encoding binary data in a binarization table when the prediction mode according to Embodiment 5 is binarized and encoded. The binarization table in Figure 79 is an example of binarizing the prediction mode values ​​using truncated unary code when the number of prediction modes M=5.

[0468] This allows for improved encoding efficiency by reducing the number of encoding tables while switching encoding tables according to the bit position of the binary data. Furthermore, when encoding the remaining bit, it is also possible to switch encoding tables for each bit and perform arithmetic encoding, or to switch the encoding table according to the arithmetic encoding result and decode.

[0469] When encoding predicted mode values ​​by binarizing them using a truncated unary code with a predicted mode number M, the predicted mode number M used in the truncated unary code may be appended to the bitstream header, etc., so that the predicted mode can be identified from the decoded binary data on the decoding side. Furthermore, the possible values ​​of the predicted mode number MaxM may be defined in the standard, etc., and the value MaxM-M (M<=MaxM) may be appended to the header. Alternatively, the predicted mode number M may not be appended to the stream, but rather defined in the profile or level of the standard, etc.

[0470] Furthermore, the predicted mode values, which have been binarized using truncated unary code, can be arithmetic coded by switching the coding table between the one-bit and remaining parts, as described above. The probability of 0 and 1 occurring in each coding table may be updated according to the actual values ​​of the binary data that occurred. Alternatively, the probability of 0 and 1 occurring in one of the coding tables may be fixed. This can reduce the processing load by suppressing the number of times the probability is updated. For example, the probability of occurrence in the one-bit part may be updated, while the probability of occurrence in the remaining bit part may be fixed.

[0471] Figure 80 is a flowchart showing an example of coding a prediction mode value according to Embodiment 5. Figure 81 is a flowchart showing an example of decoding a prediction mode value according to Embodiment 5.

[0472] As shown in Figure 80, in the coding of the predicted mode values, the predicted mode values ​​are first binarized using a truncated unary code with the number of predicted modes M (S3401).

[0473] Next, the binary data in truncated unary code is arithmetic encoded (S3402). As a result, the bitstream contains the binary data as a prediction mode.

[0474] Furthermore, as shown in Figure 81, in decoding the predicted mode values, the bitstream is first arithmetic-decoded using the number of predicted modes M to generate binary data of the truncated unary code (S3403).

[0475] Next, the prediction mode value is calculated from the binary data of the truncated unary code (S3404).

[0476] As a method for binarizing prediction mode values, we have shown an example of binarizing using a truncated unary code that uses the number of prediction modes M, but this is not necessarily the only method. For example, prediction mode values ​​may be binarized using a truncated unary code that uses the number L (L <= M) to which prediction values ​​are assigned to prediction modes. For example, when the number of prediction modes M = 5, if there are 2 surrounding 3D points that can be used to predict a 3D point to be coded, as shown in Figure 82, there may be cases where 3 prediction modes are available and the remaining 2 prediction modes are not available. For example, as shown in Figure 82, when the number of prediction modes M = 5, there may be 2 3D points around the 3D point to be coded that can be used for prediction, and no prediction values ​​have been assigned to the prediction modes that indicate prediction mode values ​​of "3" and "4".

[0477] In this case, as shown in Figure 83, by binarizing the prediction mode value using truncated unary code with the value L assigned to the prediction mode as the maximum value, it may be possible to reduce the number of bits after binarization compared to when truncated unary code is used with the number of prediction modes M. For example, in this case, L=3, so the number of bits can be reduced by binarizing with truncated unary code with a maximum value of 3. Thus, the number of bits after binarization of the prediction mode value can be reduced by binarizing with truncated unary code with the number L assigned to the prediction value as the maximum value.

[0478] The binary data after binarization may be arithmetic encoded using an encoding table. In this case, encoding efficiency may be improved by switching the encoding table for each bit of the binary data. Alternatively, to reduce the number of encoding tables, the first bit of the binarized data may be encoded using encoding table A for one bit, and each of the remaining bits (remaining bits) may be encoded using encoding table B for remaining bits. For example, when encoding the binarized data "11" with a prediction mode value of "2" shown in Figure 83, the first bit ("1") may be encoded using encoding table A, and the remaining bits ("1") may be encoded using encoding table B.

[0479] Figure 83 is a diagram illustrating an example of encoding binary data in a binarization table when the prediction mode according to Embodiment 5 is binarized and encoded. The binarization table in Figure 83 is an example of binarizing the prediction mode values ​​with a truncated unary code when the number L=3, to which prediction values ​​are assigned to the prediction mode.

[0480] This allows for improved encoding efficiency by reducing the number of encoding tables while switching encoding tables according to the bit position of the binary data. Furthermore, when encoding the remaining bit, it is also possible to switch encoding tables for each bit and perform arithmetic encoding, or to switch the encoding table according to the arithmetic encoding result and decode.

[0481] When encoding predicted mode values ​​by binarizing them using a truncated unary code with a number L assigned to each predicted value, the number L can be calculated by assigning predicted values ​​to the predicted mode in the same way as during encoding, so that the predicted mode can be identified from the decoded binary data on the decoding side. The calculated L can then be used to decode the predicted mode.

[0482] Furthermore, the predicted mode values, which have been binarized using truncated unary code, can be arithmetic coded by switching the coding table between the one-bit and remaining parts, as described above. The probability of 0 and 1 occurring in each coding table may be updated according to the actual values ​​of the binary data that occurred. Alternatively, the probability of 0 and 1 occurring in one of the coding tables may be fixed. This can reduce the processing load by suppressing the number of times the probability is updated. For example, the probability of occurrence in the one-bit part may be updated, while the probability of occurrence in the remaining bit part may be fixed.

[0483] Figure 84 is a flowchart showing another example of coding the predicted mode value according to Embodiment 5. Figure 85 is a flowchart showing another example of decoding the predicted mode value according to Embodiment 5.

[0484] As shown in Figure 84, in the coding of prediction mode values, first, the number L assigned to the prediction mode is calculated (S3411).

[0485] Next, the predicted mode values ​​are binarized using a truncated unary code with a number L (S3412).

[0486] Next, the binary data of the truncated unary code is arithmetically coded (S3413).

[0487] Furthermore, as shown in Figure 85, in decoding the prediction mode values, first, the number L assigned to the prediction mode is calculated (S3414).

[0488] Next, the bitstream is arithmetic-decoded using the number L to generate binary data of the truncated unary code (S3415).

[0489] Next, the prediction mode value is calculated from the binary data of the truncated unary code (S3416).

[0490] Prediction mode values ​​do not need to be added to every attribute value. For example, if a certain condition is met, the prediction mode may be fixed and the prediction mode value may not be added to the bitstream, while if a certain condition is not met, a prediction mode may be selected and the prediction mode value may be added to the bitstream. For example, if condition A is met, the prediction mode value may be fixed to "0" and the prediction value may be calculated from the average value of the surrounding three-dimensional points, while if condition A is not met, one prediction mode may be selected from multiple prediction modes and a prediction mode value indicating the selected prediction mode may be added to the bitstream.

[0491] One condition A is that, for example, the maximum absolute difference value maxdiff is calculated for the attribute values ​​(a[0]~a[N-1]) of N surrounding (encoded and decoded) three-dimensional points of the three-dimensional point to be encoded, and that the maximum absolute difference value maxdiff is less than the threshold Thfix. If the maximum absolute difference value of the attribute values ​​of the surrounding three-dimensional points is less than the threshold Thfix, it is determined that there is little difference in the attribute values ​​of each three-dimensional point, and that there is no difference in the predicted value even if a prediction mode is selected. By fixing the prediction mode value to "0" and not encoding the prediction mode value, it becomes possible to generate an appropriate predicted value while reducing the amount of code required to encode the prediction mode.

[0492] The threshold Thfix may be added to the bitstream header, etc., and the encoder may be configured to allow encoding by changing the value of the threshold Thfix. For example, when encoding at a high bitrate, the encoder may add a smaller value of the threshold Thfix to the header than when encoding at a low bitrate, and increase the number of cases in which the prediction mode is selected for encoding, thereby reducing the prediction residual as much as possible. Alternatively, when encoding at a low bitrate, the encoder may add a larger value of the threshold Thfix to the header than when encoding at a high bitrate, and encode with the prediction mode fixed. In this way, by increasing the number of cases in which the prediction mode is fixed for encoding at low bitrates, the amount of bits encoded for the prediction mode can be reduced while improving encoding efficiency. Furthermore, the threshold Thfix may not be added to the bitstream but may be defined in the profile or level of the standard.

[0493] The N three-dimensional points surrounding the three-dimensional point to be encoded, used for prediction, are the N encoded and decoded three-dimensional points whose distance from the three-dimensional point to be encoded is less than the threshold THd. The maximum value of N may be appended to the bitstream as NumNeighborPoint. The value of N does not always have to match the value of NumNeighborPoint, for example, if the number of surrounding encoded and decoded three-dimensional points is less than the value of NumNeighborPoint.

[0494] The example shown involves fixing the prediction mode value to "0" if the maximum absolute difference value maxdiff (the difference between the attribute values ​​of three-dimensional points surrounding the three-dimensional point to be encoded), which is used for prediction, is smaller than the threshold Thfix[i]. However, this is not necessarily the only option; the prediction mode value can be fixed to any value between "0" and "M-1". Alternatively, the fixed prediction mode value can be appended to the bitstream.

[0495] Figure 86 is a flowchart showing an example of the process for determining whether to fix the predicted mode value according to condition A during encoding according to Embodiment 5. Figure 87 is a flowchart showing an example of the process for determining whether to fix the predicted mode value to a fixed value or decode it according to condition A during decoding according to Embodiment 5.

[0496] As shown in Figure 86, first, the three-dimensional data encoding device calculates the maximum absolute difference value maxdiff for attribute values ​​at N three-dimensional points surrounding the three-dimensional point to be encoded (S3421).

[0497] Next, the three-dimensional data encoding device determines whether the maximum absolute difference value maxdiff is less than the threshold Thfix (S3422). The threshold Thfix may be encoded and added to the stream header, etc.

[0498] The three-dimensional data encoding device determines the prediction mode value to "0" (S3423) if the maximum absolute difference value maxdiff is less than the threshold Thfix (Yes in S3422).

[0499] On the other hand, if the maximum absolute difference value maxdiff is greater than or equal to the threshold Thfix (No in S3422), the three-dimensional data encoding device selects one prediction mode from among multiple prediction modes (S3424). Details of the prediction mode selection process will be described later using Figure 94.

[0500] The three-dimensional data encoding device then arithmetically encodes the predicted mode value, which indicates the selected prediction mode (S3425). Specifically, the three-dimensional data encoding device arithmetically encodes the predicted mode value by performing steps S3401 and S3402 as described in Figure 80. Alternatively, the three-dimensional data encoding device may arithmetically encode the predicted mode PredMode by binarizing it with a truncated unary code using the number of predicted modes to which the predicted value has been assigned. In other words, the three-dimensional data encoding device may arithmetically encode the predicted mode value by performing steps S3411 to S3413 as described in Figure 84.

[0501] The three-dimensional data encoding device calculates the predicted value for the prediction mode determined in step S3423 or the prediction mode selected in step S3425, and outputs the calculated predicted value (S3426). If the three-dimensional data encoding device uses the prediction mode value determined in step S3423, it calculates the average of the attribute values ​​of the surrounding N three-dimensional points as the predicted value for the prediction mode indicated by the prediction mode value "0".

[0502] Furthermore, as shown in Figure 87, the three-dimensional data decoding device first calculates the maximum absolute difference value maxdiff for attribute values ​​at N three-dimensional points surrounding the three-dimensional point to be decoded (S3431). Note that the maximum absolute difference value maxdiff may be calculated as shown in Figure 88. The maximum absolute difference value maxdiff is, for example, the maximum value among the multiple absolute values ​​calculated by calculating the absolute difference for each pair of all possible pairs when two of the N surrounding three-dimensional points are selected as a pair.

[0503] Next, the three-dimensional data decoder determines whether the maximum absolute difference value maxdiff is less than the threshold Thfix (S3432). The threshold Thfix may be set after decoding the stream header, etc.

[0504] The three-dimensional data decoder determines the prediction mode value to "0" (S3433) if the maximum absolute difference value maxdiff is less than the threshold Thfix (Yes in S3432).

[0505] On the other hand, the three-dimensional data decoder decodes the predicted mode value from the bitstream (S3434) if the maximum absolute difference value maxdiff is greater than or equal to the threshold Thfix (No in S3432).

[0506] The three-dimensional data decoder calculates the predicted value of the predicted mode indicated by the predicted mode value determined in step S3433, or the predicted mode value decoded in step S3434, and outputs the calculated predicted value (S3435). If the three-dimensional data decoder uses the predicted mode value determined in step S3433, it calculates the average of the attribute values ​​of the surrounding N three-dimensional points as the predicted value of the predicted mode indicated by the predicted mode value "0".

[0507] Figure 89 shows an example of syntax according to Embodiment 5. NumLoD, NumNeighborPoint[i], NumPredMode[i], Thfix[i], and NumOfPoint[i] in the syntax of Figure 89 will be explained in order.

[0508] NumLoD indicates the number of levels of LoD.

[0509] NumNeighborPoint[i] indicates the upper limit value of the number of surrounding points used to generate the predicted value of the three-dimensional points belonging to level i. When the number of surrounding points M is less than NumNeighborPoint[i] (M < NumNeighborPoint[i]), the predicted value may be calculated using M surrounding points. Also, when it is not necessary to divide the value of NumNeighborPoint[i] for each LoD, one NumNeighborPoint may be added to the header.

[0510] NumPredMode[i] indicates the total number (M) of prediction modes used for predicting the attribute value of level i. The possible value MaxM of the number of prediction modes is defined by a standard or the like, and the value MaxM - M (0 < M <= MaxM) may be added to the header as NumPredMode[i] and encoded by binarization with a truncated unary code of the maximum value MaxM - 1. Also, the number of prediction modes NumPredMode[i] may be defined by a profile or level of a standard or the like without being added to the stream. Also, the number of prediction modes may be defined by NumNeighborPoint[i] + NumPredMode[i]. Also, when it is not necessary to divide the value of NumPredMode[i] for each LoD, one NumPredMode may be added to the header.

[0511] Thfix[i] indicates the threshold value of the maximum absolute difference value for determining whether to fix the prediction mode of level i. If the maximum absolute difference value of the attribute values of the surrounding three-dimensional points used for prediction is less than Thfix[i], the prediction mode is fixed to 0. Note that Thfix[i] may be defined by a profile or level of a standard or the like without being added to the stream. Also, when it is not necessary to divide the value of Thfix[i] for each LoD, one Thfix may be added to the header.

[0512] NumOfPoint[i] indicates the number of three-dimensional points belonging to hierarchy i. Note that if the total number of three-dimensional points, AllNumOfPoint, is added to a separate header, NumOfPoint[NumLoD-1] (the number of three-dimensional points belonging to the lowest level) is not added to the header.

number

[0513] As an example of setting NumPredMode[i], the higher the layer where predictions are less accurate due to the greater distance between three-dimensional points belonging to the LoD, the larger the value of NumPredMode[i] can be set to increase the number of selectable prediction modes. Conversely, the lower the layer where predictions are more accurate, the smaller the value of NumPredMode[i] can be set to suppress the amount of bits required to encode the prediction mode. By setting these parameters, the higher the layer can increase the number of selectable prediction modes and reduce the prediction residual, while the lower the layer can reduce the amount of code used for the prediction mode, thereby improving encoding efficiency.

[0514] Furthermore, as an example of setting Thfix[i], the value of Thfix[i] may be set smaller for higher layers where prediction is difficult due to the greater distance between three-dimensional points belonging to the LoD, thereby increasing the number of cases in which the prediction mode is selected. Conversely, the value of Thfix[i] may be set larger for lower layers where prediction is easier, fixing the prediction mode and suppressing the amount of bits required for encoding the prediction mode. By using these settings, the prediction residual can be reduced by increasing the number of cases in which the prediction mode is selected in the higher layers, and the amount of code used in the prediction mode can be reduced by fixing the prediction mode in the lower layers, thereby improving encoding efficiency.

[0515] The above NumLoD, NumNeighborPoint[i], NumPredMode[i], Thfix[i], or NumOfPoint[i] may be entropy encoded and appended to the header. For example, each value may be binarized and arithmetic encoded. Alternatively, each value may be encoded with a fixed length to reduce processing load.

[0516] Figure 90 shows an example of syntax according to Embodiment 5. PredMode, n-bit code, and remaining code in the syntax of Figure 90 will be explained in order.

[0517] PredMode indicates the prediction mode for encoding and decoding the attribute value of the j-th 3D point in hierarchy i. PredMode takes values ​​from "0" to "M-1" (where M is the total number of prediction modes). If PredMode is not present in the bitstream (i.e., the condition maxdiff >= Thfix[i] && NumPredMode[i] > 1 is not satisfied), PredMode may be estimated to be 0. Note that PredMode is not limited to "0"; any value from "0" to "M-1" may be used as the estimated value. Furthermore, if PredMode is not present in the bitstream, the estimated value may be added separately to the header or elsewhere. Additionally, PredMode may be arithmetic encoded by binarizing it using a truncated unary code with the number of prediction modes to which the predicted value has been assigned.

[0518] The n-bit code represents the encoded data of the predicted residual of the attribute information value. The bit length of the n-bit code depends on the value of R_TH[i]. For example, if the value of R_TH[i] is 63, the bit length of the n-bit code will be 6 bits, and if the value of R_TH[i] is 255, it will be 8 bits.

[0519] The remaining code represents the encoded data of the predicted residual of the attribute information value, specifically the data encoded using Exponential-Golomb. The remaining code is decoded when the n-bit code is the same as R_TH[i], and the predicted residual is decoded by adding the value of the n-bit code and the value of the remaining code. If the n-bit code is not the same as R_TH[i], the remaining code does not need to be decoded.

[0520] The following describes the processing flow in the three-dimensional data encoding device. Figure 91 is a flowchart of the three-dimensional data encoding process by the three-dimensional data encoding device according to Embodiment 5.

[0521] First, the three-dimensional data encoding device encodes the positional information (geometry) (S3441). For example, three-dimensional data encoding is performed using an octave tree representation.

[0522] The three-dimensional data encoding device reassigns the attribute information of the original three-dimensional point to the changed three-dimensional point if the position of the three-dimensional point changes due to quantization or the like after encoding the position information (S3442). For example, the three-dimensional data encoding device performs the reassignment by interpolating the attribute information value according to the amount of change in position. For example, the three-dimensional data encoding device detects N three-dimensional points that are close to the changed three-dimensional position and performs a weighted average of the attribute information values ​​of the N three-dimensional points. For example, in the weighted average, the three-dimensional data encoding device determines the weights based on the distance from the changed three-dimensional position to each of the N three-dimensional points. Then, the three-dimensional data encoding device determines the value obtained by the weighted average as the attribute information value of the changed three-dimensional point. Furthermore, if two or more three-dimensional points change to the same three-dimensional position due to quantization or the like, the three-dimensional data encoding device may assign the average value of the attribute information of the two or more three-dimensional points before the change as the attribute information value of the changed three-dimensional point.

[0523] Next, the three-dimensional data encoding device encodes the reassigned attribute information (Attribute) (S3443). For example, if the three-dimensional data encoding device encodes multiple types of attribute information, it may encode the multiple types of attribute information sequentially. For example, if the three-dimensional data encoding device encodes color and reflectance as attribute information, it may generate a bitstream in which the encoded result of reflectance is appended after the encoded result of color. Note that the order of the multiple encoded results of attribute information appended to the bitstream is not limited to this order and may be any order.

[0524] Furthermore, the three-dimensional data encoding device may add information to the header or elsewhere indicating the starting location of the encoded data for each attribute information within the bitstream. This allows the three-dimensional data decoding device to selectively decode the attribute information that needs to be decoded, thus omitting the decoding process for attribute information that does not need to be decoded. Therefore, the processing load of the three-dimensional data decoding device can be reduced. In addition, the three-dimensional data encoding device may encode multiple types of attribute information in parallel and integrate the encoding results into a single bitstream. This allows the three-dimensional data encoding device to encode multiple types of attribute information at high speed.

[0525] Figure 92 is a flowchart of the attribute information encoding process (S3443) according to Embodiment 5. First, the three-dimensional data encoding device sets the Level of Data (LoD) (S3451). In other words, the three-dimensional data encoding device assigns each three-dimensional point to one of several LoDs.

[0526] Next, the three-dimensional data encoding device starts a loop for each Level of Data (LoD) (S3452). In other words, the three-dimensional data encoding device repeats the process from steps S3453 to S3461 for each LoD.

[0527] Next, the three-dimensional data encoding device starts a loop for each three-dimensional point (S3453). In other words, the three-dimensional data encoding device repeats the process from steps S3454 to S3460 for each three-dimensional point.

[0528] First, the three-dimensional data encoding device searches for multiple surrounding points, which are three-dimensional points that exist around the target three-dimensional point, to be used in calculating the predicted value of the target three-dimensional point to be processed (S3454).

[0529] Next, the three-dimensional data encoding device calculates a predicted value P for the target three-dimensional point (S3455). A specific example of the calculation process for the predicted value P will be described later using Figure 93.

[0530] Next, the three-dimensional data encoding device calculates the predicted residual, which is the difference between the attribute information of the target three-dimensional point and the predicted value (S3456).

[0531] Next, the three-dimensional data coding device calculates the quantized value by quantizing the prediction residual (S3457).

[0532] Next, the three-dimensional data encoding device arithmetically encodes the quantized values ​​(S3458).

[0533] Furthermore, the three-dimensional data encoding device calculates the inverse quantized value by inverse quantizing the quantized value (S3459).

[0534] Next, the three-dimensional data encoding device generates a decoded value by adding the predicted value to the inverse quantized value (S3460).

[0535] Next, the three-dimensional data encoding device terminates the loop for each three-dimensional point (S3461).

[0536] Furthermore, the three-dimensional data encoding device terminates the loop at the LoD level (S3462).

[0537] Figure 93 is a flowchart of the calculation process (S3455) for predicted values ​​in the three-dimensional data coding device according to Embodiment 5.

[0538] First, the three-dimensional data encoding device calculates a weighted average of the attribute values ​​of N three-dimensional points surrounding the target three-dimensional point, which can be used to predict the predicted value of the target three-dimensional point to be processed, and assigns the calculated weighted average to a prediction mode with a prediction mode value of "0" (S3420).

[0539] Next, the three-dimensional data encoding device outputs predicted values ​​for the target three-dimensional point by executing steps S3421 to S3426 as described in Figure 86.

[0540] Figure 94 is a flowchart of the prediction mode selection process (S3424) according to Embodiment 5.

[0541] First, the three-dimensional data encoding device assigns prediction mode values ​​to N three-dimensional points surrounding the target three-dimensional point, incrementing by 1 from "1" to "N" in order of proximity to the target three-dimensional point (S3427). Since prediction mode values ​​are assigned from "0" to "N", a total of N+1 prediction modes are generated. If N+1 exceeds the maximum number of prediction modes M (NumPredMode) that can be added to the bitstream, the three-dimensional data encoding device may generate up to M prediction modes.

[0542] Next, the three-dimensional data encoding device calculates the cost of each prediction mode and selects the prediction mode with the minimum cost (S3428).

[0543] Figure 95 is a flowchart of the process (S3428) for selecting the prediction mode that minimizes costs according to Embodiment 5.

[0544] First, the three-dimensional data encoding device is set to initial values ​​i=0 and mincost=∞ (S3471). The set initial values ​​of i and mincost are stored in the memory of the three-dimensional data encoding device.

[0545] Next, the three-dimensional data encoding device calculates the cost [i] of the i-th prediction mode PredMode[i] using, for example, equation D1 (S3472).

[0546] Next, the three-dimensional data encoding device determines whether the calculated cost cost[i] is smaller than mincost stored in memory (S3473).

[0547] Next, if the calculated cost cost[i] is smaller than the mincost stored in memory (Yes in S3473), the three-dimensional data encoding device sets mincost=cost[i], sets the prediction mode to predmode[i] (S3474), and proceeds to step S3475. In other words, it updates the value of mincost stored in memory to the value of cost[i], and stores predmode[i] in memory as the prediction mode.

[0548] On the other hand, if the calculated cost[i] is greater than or equal to the mincost stored in memory (No in S3473), the three-dimensional data encoding device proceeds to step S3475.

[0549] Next, the three-dimensional data encoding device increments the value of i by one (S3475).

[0550] Next, the three-dimensional data encoding device determines whether i is less than the number of predicted modes (S3476).

[0551] The three-dimensional data encoding device returns to step S3472 if i is less than the number of prediction modes (Yes in S3476), and terminates the process of selecting the prediction mode with the minimum cost if i is not less than the number of prediction modes (No in S3476).

[0552] The following describes the processing flow in the three-dimensional data decoding device. Figure 96 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding device according to Embodiment 5. First, the three-dimensional data decoding device decodes the position information (geometry) from the bitstream (S3444). For example, the three-dimensional data decoding device performs decoding using an octave tree representation.

[0553] Next, the three-dimensional data decoder decodes attribute information from the bitstream (S3445). For example, if the three-dimensional data decoder decodes multiple types of attribute information, it may decode the multiple types of attribute information in order. For example, if the three-dimensional data decoder decodes color and reflectance as attribute information, it decodes the color encoding result and the reflectance encoding result in the order in which they are added to the bitstream. For example, if the reflectance encoding result is added after the color encoding result in the bitstream, the three-dimensional data decoder decodes the color encoding result, and then decodes the reflectance encoding result. The three-dimensional data decoder may decode the encoding results of the attribute information added to the bitstream in any order.

[0554] Furthermore, the three-dimensional data decoding device may obtain information indicating the start location of the encoded data for each attribute information within the bitstream by decoding the header, etc. This allows the three-dimensional data decoding device to selectively decode the attribute information that needs to be decoded, thus omitting the decoding process for attribute information that does not need to be decoded. Therefore, the processing load of the three-dimensional data decoding device can be reduced. In addition, the three-dimensional data decoding device may decode multiple types of attribute information in parallel and integrate the decoding results into a single three-dimensional point cloud. This allows the three-dimensional data decoding device to decode multiple types of attribute information at high speed.

[0555] Figure 97 is a flowchart of the attribute information decoding process (S3445) according to Embodiment 5. First, the three-dimensional data decoding device sets the Level of Direction (LoD) (S3481). That is, the three-dimensional data decoding device assigns each of the multiple three-dimensional points having decoded position information to one of the multiple LoDs. For example, this assignment method is the same as the assignment method used in the three-dimensional data encoding device.

[0556] Next, the three-dimensional data decoding device starts a loop for each Level of Data (LoD) (S3482). In other words, the three-dimensional data decoding device repeats the process from steps S3483 to S3489 for each LoD.

[0557] Next, the three-dimensional data decoding device starts a loop for each three-dimensional point (S3483). In other words, the three-dimensional data decoding device repeats the process from steps S3484 to S3488 for each three-dimensional point.

[0558] First, the three-dimensional data decoding device searches for multiple surrounding points, which are three-dimensional points that exist around the target three-dimensional point, to be used in calculating the predicted value of the target three-dimensional point to be processed (S3484). This process is the same as the process in the three-dimensional data encoding device.

[0559] Next, the three-dimensional data decoding device calculates the predicted value P of the target three-dimensional point (S3485).

[0560] Next, the three-dimensional data decoding device arithmetically decodes the quantized values ​​from the bitstream (S3486).

[0561] Furthermore, the three-dimensional data decoding device calculates the inverse quantized value by inverse quantizing the decoded quantized value (S3487).

[0562] Next, the three-dimensional data decoding device generates the decoded value by adding the predicted value to the inverse quantized value (S3488).

[0563] Next, the three-dimensional data decoding device terminates the loop for each three-dimensional point (S3489).

[0564] Furthermore, the three-dimensional data decoding device terminates the loop at the LoD level (S3490).

[0565] Figure 98 is a flowchart of the calculation process (S3485) for the predicted value in the three-dimensional data decoding device according to Embodiment 5.

[0566] First, the three-dimensional data decoding device calculates a weighted average of the attribute values ​​of N three-dimensional points surrounding the target three-dimensional point, which can be used to predict the predicted value of the target three-dimensional point to be processed, and assigns the calculated weighted average to the prediction mode with a prediction mode value of "0" (S3430).

[0567] Next, the three-dimensional data decoding device outputs predicted values ​​for the target three-dimensional points by executing steps S3431 to S3435 as described in Figure 87.

[0568] Alternatively, instead of executing step S3430, if the result is determined to be Yes in step S3432, or if the decoded prediction mode value in step S3434 is "0", the weighted average of the attribute values ​​of N three-dimensional points surrounding the target three-dimensional point can be calculated as the prediction value, which can be used to predict the prediction value of the target three-dimensional point to be processed. This eliminates the need to calculate the average value for prediction modes other than the prediction mode indicated by the prediction mode value of "0", thereby reducing the amount of processing required.

[0569] Figure 99 is a flowchart of the decoding process (S3434) of the prediction mode according to Embodiment 5.

[0570] First, the three-dimensional decoder assigns prediction mode values ​​to the N three-dimensional points surrounding the target three-dimensional point, incrementing by 1 from "1" to "N" in order of proximity to the target three-dimensional point (S3491). Since prediction mode values ​​are assigned from "0" to "N", a total of N+1 prediction modes are generated. If N+1 exceeds the maximum number of prediction modes M (NumPredMode) that can be added to the bitstream, the three-dimensional data decoder may generate up to M prediction modes.

[0571] Next, the three-dimensional data decoding device arithmetically decodes the predicted modes using the number of predicted modes (S3492). Specifically, the three-dimensional data decoding device may arithmetically decode the predicted modes by performing steps S3403 and S3404 as described in Figure 81. Alternatively, the three-dimensional data decoding device may arithmetically decode the predicted modes by performing steps S3414 to S3416 as described in Figure 85.

[0572] Figure 100 is a block diagram showing the configuration of the attribute information encoding unit 3400 included in the three-dimensional data encoding device in Embodiment 5. Figure 100 also shows the details of the attribute information encoding unit, which is one of the three-dimensional data encoding devices that includes a location information encoding unit and an attribute information reassignment unit.

[0573] The attribute information coding unit 3400 includes an LoD generation unit 3401, a surrounding search unit 3402, a prediction unit 3403, a prediction residual calculation unit 3404, a quantization unit 3405, an arithmetic coding unit 3406, an inverse quantization unit 3407, a decoded value generation unit 3408, and a memory 3409.

[0574] The LoD generation unit 3401 generates LoDs using three-dimensional point position information (geometry).

[0575] The surrounding search unit 3402 uses the LoD generation result from the LoD generation unit 3401 and distance information indicating the distance between each three-dimensional point to search for neighboring three-dimensional points adjacent to each three-dimensional point.

[0576] The prediction unit 3403 generates predicted values ​​for the attribute information of the target three-dimensional point to be encoded. Specifically, the prediction unit 3403 assigns predicted values ​​to prediction modes indicated by prediction mode values ​​from "0" to "M-1" and selects a prediction mode. The prediction unit 3403 outputs the selected prediction mode, specifically the prediction mode value indicating the prediction mode, to the arithmetic coding unit. The prediction unit 3403 then performs the processing in step S3455, for example.

[0577] The prediction residual calculation unit 3404 calculates (generates) the prediction residual of the predicted value of the attribute information generated by the prediction unit 3403. The prediction residual calculation unit 3404 then performs the process in step S3456.

[0578] The quantization unit 3405 quantizes the predicted residuals of the attribute information calculated by the predicted residual calculation unit 3404.

[0579] The arithmetic coding unit 3406 arithmetically codes the predicted residuals after quantization by the quantization unit 3405. The arithmetic coding unit 3406 outputs the bitstream containing the arithmetic coded predicted residuals to, for example, a three-dimensional data decoding device.

[0580] The prediction residual may be binarized, for example, by the quantization unit 3405 before being arithmetically encoded by the arithmetic encoding unit 3406. The arithmetic encoding unit 3406 may also generate and encode various header information. Furthermore, the arithmetic encoding unit 3406 may obtain the prediction mode used for encoding from the prediction block, arithmetic encode it, and add it to the bitstream.

[0581] The inverse quantization unit 3407 inversely quantizes the predicted residual after it has been quantized by the quantization unit 3405. The inverse quantization unit 3407 then performs the process in step S3459.

[0582] The decoded value generation unit 3408 generates a decoded value by adding the predicted value of the attribute information generated by the prediction unit 3403 and the predicted residual after inverse quantization by the inverse quantization unit 3407.

[0583] Memory 3409 is a memory that stores the decoded values ​​of the attribute information of each three-dimensional point decoded by the decoded value generation unit 3408. For example, when the prediction unit 3403 generates prediction values ​​for three-dimensional points that have not yet been encoded, it uses the decoded values ​​of the attribute information of each three-dimensional point stored in memory 3409 to generate the prediction values.

[0584] Figure 101 is a block diagram showing the configuration of the attribute information decoding unit 3410 of the three-dimensional data decoding device according to Embodiment 5. Figure 101 also shows the details of the attribute information decoding unit, which is one of the two parts of the three-dimensional data decoding device that make up the attribute information decoding unit and the position information decoding unit.

[0585] The attribute information decoding unit 3410 includes an LoD generation unit 3411, a surrounding search unit 3412, a prediction unit 3413, an arithmetic decoding unit 3414, an inverse quantization unit 3415, a decoded value generation unit 3416, and a memory 3417.

[0586] The LoD generation unit 3411 generates LoDs using the position information (geometry information) of three-dimensional points decoded by the position information decoding unit (not shown).

[0587] The surrounding search unit 3412 uses the LoD generation result from the LoD generation unit 3411 and distance information indicating the distance between each three-dimensional point to search for neighboring three-dimensional points adjacent to each three-dimensional point.

[0588] The prediction unit 3413 generates predicted values ​​for the attribute information of the target three-dimensional point to be decoded. The prediction unit 3413 performs, for example, the process in step S3485.

[0589] The arithmetic decoding unit 3414 arithmetically decodes the prediction residuals in the bitstream obtained from the attribute information coding unit 3400. The arithmetic decoding unit 3414 may also decode various header information. Alternatively, the arithmetic decoding unit 3414 may output the arithmetic-decoded prediction mode to the prediction unit 3413. In this case, the prediction unit 3413 may calculate the prediction value using the prediction mode obtained by arithmetic decoding in the arithmetic decoding unit 3414.

[0590] The inverse quantization unit 3415 inversely quantizes the predicted residuals that were arithmetically decoded by the arithmetic decoding unit 3414.

[0591] The decoded value generation unit 3416 generates a decoded value by adding the predicted value generated by the prediction unit 3413 and the predicted residual after inverse quantization by the inverse quantization unit 3415. The decoded value generation unit 3416 outputs the decoded attribute information data to another device.

[0592] Memory 3417 is a memory that stores the decoded values ​​of the attribute information of each three-dimensional point decoded by the decoded value generation unit 3416. For example, when the prediction unit 3413 generates prediction values ​​for three-dimensional points that have not yet been decoded, it uses the decoded values ​​of the attribute information of each three-dimensional point stored in memory 3417 to generate the prediction values.

[0593] (Embodiment 6) As described above, the three-dimensional data encoding device calculates the maximum absolute difference value of attribute values ​​in N three-dimensional points (surrounding three-dimensional points) surrounding the target three-dimensional point to be encoded (i.e., the maximum absolute value of the difference in attribute values ​​of any two of the N three-dimensional points), which can be used for prediction in both the three-dimensional data encoding device and the three-dimensional data decoding device. The three-dimensional data encoding device also fixes the prediction mode according to the calculated value, that is, it switches between using a predetermined prediction mode or selecting a prediction mode and adding it to the bitstream.

[0594] However, the three-dimensional data encoding device does not need to switch between fixing the prediction mode or selecting a prediction mode and adding it to the bitstream, depending on the maximum absolute difference value. For example, the three-dimensional data encoding device may choose whether to fix the prediction mode or select a prediction mode under the above conditions, and add the result to the bitstream as a prediction mode fixed flag.

[0595] This allows the three-dimensional data decoder to determine whether the three-dimensional data encoder fixed the prediction mode or selected a prediction mode and encoded it, by decoding the prediction mode fixed flag attached to the bitstream.

[0596] The three-dimensional data decoder can determine that if the prediction mode is fixed by the three-dimensional data encoder, the prediction mode is not encoded in the bitstream. On the other hand, if the prediction mode is selected by the three-dimensional data encoder, the three-dimensional data decoder can determine that it is necessary to decode the prediction mode in the bitstream. As a result, the three-dimensional data decoder can correctly decode the encoded prediction mode contained in the bitstream.

[0597] Furthermore, this allows the 3D data decoder to arithmetically decode the prediction mode without having to calculate the maximum absolute difference value of attribute values ​​at N 3D points surrounding the target 3D point to be encoded, which can then be used for prediction. This enables the 3D data decoder to perform arithmetic decoding of the bitstream and LoD generation, etc., in parallel. As a result, the 3D data decoder can improve the overall throughput of the processing.

[0598] Furthermore, the three-dimensional data encoding device may add a prediction mode fixed flag to each three-dimensional point.

[0599] This allows the 3D data decoding device to switch between fixing or selecting a prediction mode for each 3D point based on a prediction mode fixing flag. As a result, the 3D data encoding device can improve encoding efficiency.

[0600] Furthermore, the three-dimensional data encoding device may set a prediction mode fixing flag for each Level of Data (LoD). For example, the three-dimensional data encoding device may set the prediction mode fixing flag to 0 for the upper layers of the LoD where predictions are less likely to be accurate, allowing the user to select a prediction mode, while setting the prediction mode fixing flag to 1 for the lower layers where predictions are more likely to be accurate, fixing the prediction mode. This allows the three-dimensional data encoding device to reduce the amount of code that needs to be assigned to the prediction mode.

[0601] Figure 102 is a flowchart showing the process of determining the prediction mode performed by the three-dimensional data encoding device according to this embodiment.

[0602] First, the three-dimensional data encoding device calculates the maximum absolute difference value maxdiff for attribute values ​​at N three-dimensional points surrounding the target three-dimensional point to be encoded (S3501).

[0603] Figure 103 shows an example of the syntax for the prediction mode determination process performed by the three-dimensional data encoding device according to this embodiment. Specifically, Figure 103 shows an example of the syntax for step S3501 in Figure 102.

[0604] In Figure 103, the attribute values ​​around the target three-dimensional point to be encoded are shown as a[0] to a[N-1], and the maximum absolute difference value is shown as maxdiff. Here, the N attribute values ​​around the target three-dimensional point to be encoded are attribute values ​​that have been encoded by a three-dimensional data encoding device and decoded by a three-dimensional data decoding device.

[0605] The three-dimensional data encoding device calculates the maximum absolute difference value maxdiff using, for example, the syntax example shown in Figure 103.

[0606] Furthermore, the N three-dimensional points surrounding the target three-dimensional point to be encoded, which the three-dimensional data encoding device uses for prediction, are the N encoded and decoded three-dimensional points whose distance from the target three-dimensional point to be encoded is less than the threshold THd.

[0607] Here, as described above, the three-dimensional data encoding device may add the maximum value of N as NumNeighborPoint to the bit stream. Note that when the number of encoded and decoded three-dimensional points in the attribute information of the three-dimensional points around the target three-dimensional points to be encoded is less than the value of NumNeighborPoint, etc., the value of N may not match the value of NumNeighborPoint.

[0608] Referring to FIG. 102 again, next, the three-dimensional data encoding device determines whether maxdiff < Thfix is satisfied (S3502). Note that Thfix is a constant arbitrarily determined in advance.

[0609] When the three-dimensional data encoding device determines that maxdiff < Thfix is satisfied (Yes in S3502), it sets the prediction mode fixed flag (fixedPredMode) to 1 and performs arithmetic encoding (S3503).

[0610] Next, the three-dimensional data encoding device determines the value of the prediction mode (PredMode / hereinafter also simply referred to as the prediction mode) to be 0 (S3504).

[0611] On the other hand, when the three-dimensional data encoding device determines that maxdiff < Thfix is not satisfied (No in S3502), it sets the prediction mode fixed flag to 0 and performs arithmetic encoding (S3505).

[0612] Next, the three-dimensional data encoding device selects the prediction mode (S3506).

[0613] Next, the three-dimensional data encoding device performs arithmetic encoding on the selected prediction mode (S3507).

[0614] In the above example, the 3D data device fixed the prediction mode to 0 if the maximum absolute difference in attribute values ​​of 3D points surrounding the target 3D point to be encoded was less than Thfix[i], but this is not necessarily the only example. For example, the 3D data encoding device may be configured to fix the prediction mode to one of the ranges from 0 to M-1.

[0615] Furthermore, the three-dimensional data encoding device may add a fixed prediction mode value (also called PredMode or mode number) to the bitstream.

[0616] Figure 104 is a flowchart showing the process of determining the prediction mode performed by the three-dimensional data decoding device according to this embodiment.

[0617] First, the three-dimensional data decoding device arithmetically decodes the encoded prediction mode fixed flag contained in the bitstream (S3511).

[0618] Next, the three-dimensional data decoding device determines whether the prediction mode fixed flag == 1 is satisfied (S3512).

[0619] If the three-dimensional data decoding device determines that the prediction mode fixed flag == 1 is satisfied (Yes in S3512), it sets the prediction mode value to 0 (S3513).

[0620] On the other hand, if the three-dimensional data decoding device determines that the prediction mode fixed flag == 1 is not satisfied (No in S3512), it determines the prediction mode value by decoding it from the bitstream (S3514).

[0621] The three-dimensional data decoding device determines the predicted value based on the determined prediction mode.

[0622] The prediction mode lock flag may be placed at any position.

[0623] Figure 105 shows an example of attribute data syntax when a prediction mode fixed flag is set for each three-dimensional point.

[0624] fixedPredMode is a flag that indicates whether the 3D data encoding device fixes the prediction mode. For example, if the value of fixedPredMode is 1, the 3D data encoding device fixes the prediction mode, and if the value of fixedPredMode is 0, the 3D data encoding device selects a prediction mode.

[0625] Alternatively, fixedPredMode may be set for each level of the LoD, as shown in Figure 106.

[0626] Figure 106 shows an example of attribute data syntax when a prediction mode fixed flag is set for each level of the LoD.

[0627] fixedPredMode[i] is a flag that indicates whether to fix the prediction mode for level i of the LoD.

[0628] PredMode is a value that indicates the prediction mode for encoding and decoding the attribute value of the j-th 3D point in hierarchy i. PredMode takes values ​​from 0 to M-1 (where M is the total number of prediction modes). Note that the 3D data decoder may estimate PredMode to be 0 if PredMode is not present in the bitstream (specifically, if !fixedPredMode&&NumPredMode[i]>1 is not satisfied).

[0629] Furthermore, if PredMode is not present in the bitstream, the three-dimensional data decoder does not need to set PredMode to 0; it may use any value from 0 to M-1 as an estimated value.

[0630] Furthermore, the three-dimensional data encoding device may add an estimated value to the header or elsewhere if PredMode is not present in the bitstream.

[0631] Furthermore, as described above, the three-dimensional data encoding device may also binarize the PredMode using a truncated unary code with the total number of prediction modes M, and then arithmetic encode the binarized value.

[0632] Truncated unali coding is one method of binarization. In truncated unali coding, for multi-level signals that take values ​​other than the maximum value, a signal is generated by arranging the same number of 1s as the value represented by the multi-level signal, and then adding a 0 at the end. For multi-level signals that take the maximum value, the maximum value is set in advance, and a signal is generated by arranging the same number of 1s as the value represented by the multi-level signal (a signal without a 0 at the end). For example, as described above, if there are 5 prediction modes, when the three-dimensional data coding device binarizes each prediction mode with truncated unali coding, the prediction modes will be 0, 10, 110, 1110, and 1111 respectively for prediction modes 0 to 4. In this way, the three-dimensional data coding device binarizes the prediction modes with truncated unali coding using the number of prediction modes (i.e., according to the number of prediction modes).

[0633] In truncated unali codes, for multi-level signals that take values ​​other than the maximum value, a signal may be generated by arranging the same number of zeros as the value represented by the multi-level signal and then adding a 1 at the end. In other words, the above-mentioned 0s and 1s may be inverted.

[0634] Furthermore, the three-dimensional data encoding device may arithmetically encode the value of the total number of prediction modes M and add it to the header as NumPredMode.

[0635] Note that NumPredMode is a value that indicates the total number of prediction modes.

[0636] As a result, the 3D data decoder can decode the NumPredMode in the header to calculate the total number of predicted modes M, and then decode the PredMode using the total number of predicted modes M. Therefore, the 3D data decoder can perform arithmetic decoding of the bitstream without setting (generating) the Level of Data (LoD), calculating the 3D points around the target 3D point to be decoded that are available for prediction, and waiting for the calculation of the number of predicted modes to which predicted values ​​have been assigned. This allows the 3D data decoder to perform arithmetic decoding of the bitstream and LoD generation, etc., in parallel, thereby improving the overall throughput of the process.

[0637] Furthermore, the n-bit code is encoded data representing the predicted residual of the attribute information value. The bit length of the n-bit code depends on the value of R_TH[i]. For example, if the value of R_TH[i] is 63, it will be 6 bits, and if the value of R_TH[i] is 255, it will be 8 bits. The value of R_TH can be arbitrarily determined in advance.

[0638] Furthermore, the remaining code is the encoded data of the predicted residual of the attribute information value, encoded using exponential Golomb coding. The three-dimensional data decoder decodes the remaining code when the n-bit code is the same as R_TH[i], and decodes the predicted residual by adding the value of the n-bit code and the value of the remaining code.

[0639] Furthermore, if the n-bit code is not the same value as R_TH[i], the three-dimensional data decoder does not need to decode the remaining code.

[0640] Figure 107 is a flowchart showing an example of the coding process in the prediction mode of a three-dimensional data coding device.

[0641] First, the three-dimensional data encoding device binarizes the prediction modes using a truncated unali code with the total number of prediction modes M (S3521).

[0642] Next, the three-dimensional data encoding device arithmetically encodes the binarized data of the truncated unali code (S3522).

[0643] Next, the three-dimensional data encoding device adds the total number of prediction modes M as NumPredMode to the header and encodes it (S3523).

[0644] The three-dimensional data encoding device transmits, for example, a bitstream containing the encoded NumPredMode to the three-dimensional data decoding device.

[0645] Figure 108 is a flowchart showing an example of the decoding process in the prediction mode of a three-dimensional data decoding device.

[0646] First, the three-dimensional data decoder decodes the encoded NumPredMode contained in the bitstream to set the total number of prediction modes M (S3524).

[0647] Next, the three-dimensional data decoding device uses the total number of decoded prediction modes M to arithmetic decode the encoded PredMode and generate binarized data of the truncated unali code (S3525).

[0648] Next, the three-dimensional data decoding device calculates the prediction mode from the binarized data of the truncated unali code (S3526).

[0649] Furthermore, the three-dimensional data encoding device may arithmetically encode the fixedPredMode described above using an encoding table. Each three-dimensional data encoding device may also update the probability of occurrence of 0 and 1 in the encoding table according to the actual value of fixedPredMode that occurs. Alternatively, the three-dimensional data encoding device may fix the probability of occurrence of 0 and 1 in either of the encoding tables. This allows the three-dimensional data encoding device to reduce processing load by suppressing the number of updates required for the probability of occurrence of 0 and 1 in the encoding table.

[0650] Figure 109 is a flowchart of the three-dimensional data encoding process performed by the three-dimensional data encoding device according to this embodiment.

[0651] First, the three-dimensional data encoding device encodes the position information (geometry) (S3531). Note that, in this case, the three-dimensional data encoding device may use, for example, an octree representation for encoding.

[0652] Next, the three-dimensional data encoding device, after encoding the position information, reassigns the attribute information of the original three-dimensional point to the changed three-dimensional point if the position of the three-dimensional point changes due to quantization or the like (S3532).

[0653] Furthermore, the three-dimensional data encoding device may interpolate and reassign attribute information values ​​according to the amount of change in position. For example, the three-dimensional data encoding device may detect N three-dimensional points around the target three-dimensional point of the encoding target before the change that are close to the changed three-dimensional position, and weight the attribute information values ​​of the detected N three-dimensional points based on the distance from the target three-dimensional position of the encoding target after the change to each of the N three-dimensional points, and use that value as the attribute information value of the changed three-dimensional point. In addition, if two or more three-dimensional points change to the same three-dimensional position due to quantization or the like, the three-dimensional data encoding device may assign the average value of the attribute information of the two or more three-dimensional points before the change as the attribute information value of the changed point.

[0654] Next, the three-dimensional data encoding device encodes the attribute information after reassignment (S3533). For example, if the three-dimensional data encoding device encodes multiple attribute information, it may encode them sequentially.

[0655] Furthermore, for example, if the three-dimensional data encoding device encodes color and reflectance as attribute information, it may generate a bitstream in which the reflectance encoding result is appended after the color encoding result.

[0656] Furthermore, the three-dimensional data encoding device may add the encoded attribute information results to the bitstream in any order. The three-dimensional data encoding device may also add the starting location of the encoded data for each attribute information within the bitstream to a header or similar element.

[0657] This allows the 3D data decoding device to decode attribute information that needs to be decoded. Therefore, the 3D data decoding device can reduce processing load by omitting the decoding process for attribute information that does not need to be decoded.

[0658] Furthermore, the three-dimensional data encoding device may encode multiple attribute information in parallel and integrate the encoding results into a single bitstream.

[0659] This allows the three-dimensional data encoding device to encode multiple attribute pieces of information at high speed.

[0660] Figure 110 is a flowchart of the attribute information encoding process (S3533) shown in Figure 109.

[0661] First, the three-dimensional data encoding device sets the Level of Direction (LoD) (S35331). In other words, the three-dimensional data encoding device assigns each three-dimensional point to one of several LoDs.

[0662] Next, the three-dimensional data encoding device starts a loop for each Level of Data (LoD) (S35332). In other words, the three-dimensional data encoding device repeatedly performs the processing in steps S35333 to S35341 for each LoD.

[0663] Next, the three-dimensional data encoding device starts a loop for each three-dimensional point (S35333). In other words, the three-dimensional data encoding device repeatedly performs the processing in steps S35334 to S35340 for each three-dimensional point for a given Line of Data (LoD). Figure 110 shows the encoding process at the target three-dimensional point P to be encoded.

[0664] Next, the three-dimensional data encoding device searches for multiple surrounding points, which are three-dimensional points that exist around the target three-dimensional point P, to be used in calculating the predicted value of the target three-dimensional point P to be processed (S35334).

[0665] Next, the three-dimensional data encoding device calculates a predicted value for the target three-dimensional point P (S35335).

[0666] Next, the three-dimensional data encoding device calculates the predicted residual, which is the difference between the attribute information of the target three-dimensional point P and the predicted value (S35336).

[0667] Next, the three-dimensional data coding device calculates the quantized value by quantizing the prediction residual (S35337).

[0668] Next, the three-dimensional data encoding device arithmetically encodes the quantized values ​​(S35338).

[0669] Next, the three-dimensional data encoding device calculates the inverse quantized value by inverse quantizing the quantized value (S35339).

[0670] Next, the three-dimensional data encoding device generates a decoded value by adding the predicted value to the inverse quantized value (S35340).

[0671] Next, the three-dimensional data encoding device terminates the loop for each three-dimensional point (S35341).

[0672] Furthermore, the three-dimensional data encoding device terminates the loop for each Level of Data (S35342).

[0673] Figure 111 is a flowchart of the calculation process (S35335) for the predicted value shown in Figure 110.

[0674] First, the three-dimensional data encoding device calculates the weighted average of the attribute values ​​of N surrounding three-dimensional points of the target three-dimensional point to be encoded, which can be used for prediction, and assigns it to prediction mode 0 (S353351).

[0675] Next, the three-dimensional data encoding device calculates the maximum absolute difference value maxdiff of the attribute values of N surrounding three-dimensional points of the target three-dimensional points to be encoded (S353352).

[0676] Next, the three-dimensional data encoding device determines whether the maximum absolute difference value maxdiff < Thfix is satisfied (S353353).

[0677] When the three-dimensional data encoding device determines that the maximum absolute difference value maxdiff < Thfix is satisfied (Yes in S353353), it sets the prediction mode fixed flag to 1 and performs arithmetic encoding (S353354).

[0678] Next, the three-dimensional data encoding device determines the prediction mode to be 0 (the prediction mode indicating the weighted average value as the predicted value) (S353355).

[0679] Next, the three-dimensional data encoding device arithmetic-encodes the predicted value of the determined prediction mode and outputs it to, for example, the three-dimensional data decoding device (S353356).

[0680] On the other hand, when the three-dimensional data encoding device determines that the maximum absolute difference value maxdiff < Thfix is not satisfied (No in S353353), it sets the prediction mode fixed flag to 0 and performs arithmetic encoding (S353357).

[0681] Next, the three-dimensional data encoding device selects and determines the prediction mode (S353358).

[0682] Next, the three-dimensional data encoding device arithmetic-encodes the selected and determined prediction mode (S353359).

[0683] Note that, as described above, the three-dimensional data encoding device may binarize the prediction mode PredMode using the total number of prediction modes M in a truncated unary code and perform arithmetic encoding. Also, the three-dimensional data encoding device may encode and add the total number of prediction modes M as NumPredMode to the header.

[0684] As a result, the three-dimensional data decoder can correctly decode the prediction mode PredMode by decoding the NumPredMode in the header.

[0685] Furthermore, the 3D data encoding device does not need to encode PredMode when NumPredMode=1. This allows the 3D data encoding device to reduce the amount of code used when NumPredMode=1.

[0686] Figure 112 is a flowchart of the prediction mode selection process (S353358) shown in Figure 111.

[0687] First, the three-dimensional data encoding device assigns the attribute information of N three-dimensional points surrounding the target three-dimensional point to be encoded, in order from closest to furthest distance from the target three-dimensional point, from prediction mode 1 to N (S3541). For example, the three-dimensional data encoding device generates N+1 prediction modes. If N+1 exceeds the total number of prediction modes M (NumPredMode) added to the bitstream, the three-dimensional data encoding device may generate up to M prediction modes.

[0688] Next, the three-dimensional data encoding device calculates the cost of each mode and selects the prediction mode that minimizes the cost (S3542).

[0689] Figure 113 is a flowchart showing a specific example of the prediction mode selection process (S3542) shown in Figure 112.

[0690] First, the three-dimensional data encoding device performs i=0 and mincost=∞ (S35421).

[0691] Next, the three-dimensional data encoding device calculates the cost (cost[i]) of the i-th prediction mode, PredMode[i] (S35422).

[0692] Next, the three-dimensional data encoding device determines whether cost[i] < mincost is satisfied (S35423).

[0693] Next, when the three-dimensional data encoding device determines that cost[i] < mincost is satisfied (Yes in S35423), it sets mincost = cost[i] and sets the prediction mode to PredMode[i]. (S35424).

[0694] Next, after step S35424 or when the three-dimensional data encoding device determines that cost[i] < mincost is not satisfied (No in S35423), it performs i = i + 1 (S35425).

[0695] Next, the three-dimensional data encoding device determines whether i < the number of prediction modes (total number of prediction modes) is satisfied (S35426).

[0696] When the three-dimensional data encoding device determines that i < the number of prediction modes is not satisfied (No in S35426), it ends the selection process, and when it determines that i < the number of prediction modes is satisfied (Yes in S35426), it returns the process to step S35422.

[0697] FIG. 114 is a flowchart of three-dimensional data decoding processing by the three-dimensional data decoding device according to the present embodiment.

[0698] The three-dimensional data decoding device decodes the position information (geometry) of the encoded three-dimensional points (S3551). The three-dimensional data decoding device may decode the position information using, for example, an octree representation.

[0699] Next, the three-dimensional data decoding device decodes the attribute information of the encoded three-dimensional points (S3552).

[0700] Furthermore, if the 3D data decoding device decodes multiple attribute information, it may decode them sequentially. For example, if the 3D data decoding device decodes color and reflectance as attribute information, it may decode a bitstream in which the color encoding result is followed by the reflectance encoding result in that order.

[0701] Furthermore, the three-dimensional data decoding device may decode the encoded results of the attribute information attached to the bitstream in any order.

[0702] Furthermore, the three-dimensional data decoding device may obtain the starting location of the encoded data for each attribute information within the bitstream by decoding the header, etc.

[0703] This allows the 3D data decoding device to decode the attribute information that needs to be decoded. Therefore, the 3D data decoding device can reduce the processing load by omitting the decoding process for attribute information that does not need to be decoded.

[0704] Furthermore, the three-dimensional data decoding device may decode multiple attribute information in parallel and integrate the decoding results into a single three-dimensional point cloud.

[0705] This allows the three-dimensional data decoding device to decode multiple attribute pieces of information at high speed.

[0706] Figure 115 is a flowchart of the calculation process (S3552) for the predicted value of P shown in Figure 114.

[0707] First, the three-dimensional data decoder obtains the decoded position information of the three-dimensional points contained in the bitstream (S35520). Specifically, the three-dimensional data decoder obtains the decoded position information by decoding the encoded position information of the three-dimensional points contained in the bitstream transmitted from the three-dimensional data encoder.

[0708] Next, the 3D data decoder sets the Level of Direction (LoD) (S35521). In other words, the 3D data decoder assigns each 3D point to one of several LoDs.

[0709] Next, the three-dimensional data decoding device starts a loop for each Level of Data (LoD) (S35522). In other word...

Claims

1. A three-dimensional data encoding method for encoding three-dimensional points having a prediction tree structure, The node's position is encoded based on a method corresponding to one of one or more selectable methods used to predict the node's position. A bitstream is generated that includes mode information indicating the method used for the node, The one or more selectable methods described above are modified based on the depth of the nodes in the prediction tree structure. Three-dimensional data encoding method.

2. The mode information includes a value indicating the method used for the node, The aforementioned value is less than or equal to the value of the depth of the node in the prediction tree structure. The three-dimensional data encoding method according to claim 1.

3. In the coding described above, the predicted value of the node is calculated based on the method described above. The three-dimensional data encoding method according to claim 1.

4. In the above encoding, a prediction residual is derived that shows the difference between the node's position and the calculated predicted value of the node. The three-dimensional data encoding method according to claim 3.

5. If the depth of the node in the prediction tree structure is 0, then the number of selectable methods is 1. The three-dimensional data encoding method according to claim 3.

6. If the depth of the node in the prediction tree structure is 2, then the number of selectable methods (1 or more) is 3, or If the depth of the node in the prediction tree structure is 3, then the number of selectable methods (1 or more) is 4. The three-dimensional data encoding method according to claim 1.

7. The one or more selectable methods include a first method used for the root node of the prediction tree structure. The three-dimensional data encoding method according to claim 6.

8. The one or more selectable methods include a second method in which a node having a parent node can be used in the prediction tree structure, a third method in which a node having both a parent node and a grandparent node can be used, and a fourth method in which a node having a parent node, a grandparent node and a great-grandparent node can be used. The three-dimensional data encoding method according to claim 6.

9. A three-dimensional data decoding method for decoding three-dimensional points having a prediction tree structure, Obtain mode information indicating the method used to predict the node positions in the aforementioned prediction tree structure, The position of the node is decoded based on the method indicated by the mode information, The method indicated by the mode information corresponds to one of one or more selectable methods that are modified based on the depth of the node in the prediction tree structure. Three-dimensional data decoding method.

10. The mode information includes a value corresponding to one of the one or more selectable methods, and the value is less than or equal to the depth of the node in the prediction tree structure. The method for decoding three-dimensional data according to claim 9.

11. In the decoding, predictive values ​​used to calculate the node's position are calculated based on the method indicated by the mode information. The method for decoding three-dimensional data according to claim 9.

12. In the decoding process, the predicted residual is derived, and the node's position is calculated using the predicted value and the predicted residual. The method for decoding three-dimensional data according to claim 11.

13. If the depth of the node in the prediction tree structure is 0, then the number of selectable methods is 1. The method for decoding three-dimensional data according to claim 11.

14. If the depth of the node in the prediction tree structure is 2, then the number of selectable methods (1 or more) is 3, or If the depth of the node in the prediction tree structure is 3, then the number of selectable methods (1 or more) is 4. The method for decoding three-dimensional data according to claim 9.

15. The one or more selectable methods include a first method used for the root node of the prediction tree structure. The method for decoding three-dimensional data according to claim 14.

16. The one or more selectable methods include a second method in which a node having a parent node can be used in the prediction tree structure, a third method in which a node having both a parent node and a grandparent node can be used, and a fourth method in which a node having a parent node, a grandparent node and a great-grandparent node can be used. The method for decoding three-dimensional data according to claim 14.

17. A three-dimensional data encoding device that encodes three-dimensional points having a prediction tree structure, Processor and Equipped with memory, The processor uses the memory to: The node's position is encoded based on a method corresponding to one of one or more selectable methods used to predict the node's position. A bitstream is generated that includes mode information indicating the method used for the node, The one or more selectable methods described above are modified based on the depth of the nodes in the prediction tree structure. Three-dimensional data encoding device.

18. A three-dimensional data decoding device that decodes three-dimensional points having a prediction tree structure, Processor and Equipped with memory, The processor uses the memory to: Obtain mode information indicating the method used to predict the node positions in the aforementioned prediction tree structure, The position of the node is decoded based on the method indicated by the mode information, The method indicated by the mode information corresponds to one of one or more selectable methods that are modified based on the depth of the node in the prediction tree structure. Three-dimensional data decoding device.

Images