Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

Abstract

To provide a three-dimensional data encoding method and the like that can improve encoding efficiency.SOLUTION: Provided is a three-dimensional data encoding method for encoding three-dimensional points having a prediction tree structure, the method including encoding the positions of nodes on the basis of a method corresponding to one of one or more selectable methods used for prediction of the positions of nodes, and generating a bit stream including mode information indicating the method used for the nodes (S9784), wherein the one or more selectable methods are changed on the basis of the depth of the nodes in the prediction tree structure.SELECTED DRAWING: Figure 166

Description

[Technical Field]

[0001] The present disclosure relates to a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, and a three-dimensional data decoding device. [Background technology]

[0002] In the future, devices and services that utilize 3D data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, video distribution, etc. 3D data can be acquired in a variety of ways, including distance sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

[0003] One method of representing three-dimensional data is a point cloud, which represents the shape of a three-dimensional structure using a group of points in three-dimensional space. A point cloud stores the position and color of the points. Point clouds are expected to become the mainstream method of representing three-dimensional data, but point clouds require a very large amount of data. Therefore, when storing or transmitting three-dimensional data, data compression through encoding is essential, just as with two-dimensional video images (examples include MPEG-4 AVC or HEVC standardized by MPEG).

[0004] In addition, compression of point clouds is partially supported by public libraries that perform point cloud-related processing (Point Cloud Library).

[0005] Furthermore, a technique is known in which three-dimensional map data is used to search for and display facilities located around a vehicle (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2014 / 020663 Summary of the Invention [Problem to be solved by the invention]

[0007] It is desirable to be able to improve the coding efficiency in coding three-dimensional data.

[0008] An object of the present disclosure is to provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency. [Means for solving the problem]

[0009] A three-dimensional data encoding method according to one aspect of the present disclosure is a three-dimensional data encoding method for encoding three-dimensional points having a predictive tree structure, the method encoding the positions of the nodes based on a method corresponding to one of one or more selectable methods used to predict the positions of the nodes, and generating a bitstream including mode information indicating the method used for the nodes, the one or more selectable methods being changed based on the depth of the nodes in the predictive tree structure. , when the depth of the node in the prediction tree structure is 1, the number of the one or more selectable methods is 2. .

[0010] A three-dimensional data decoding method according to one aspect of the present disclosure includes decoding three-dimensional points having a prediction tree structure. and obtaining mode information indicating a method used to predict a node position in the prediction tree structure, and decoding the node position based on the method indicated by the mode information, the method indicated by the mode information corresponding to one of one or more selectable methods that are changed based on the depth of the node in the prediction tree structure. If the depth of the node in the prediction tree structure is 1, the number of the one or more selectable methods is 2. .

[0011] A three-dimensional data encoding method according to one embodiment of the present disclosure is a three-dimensional data encoding method for encoding a plurality of three-dimensional points having a hierarchical structure, which uses second position information of one or more second three-dimensional points surrounding a first three-dimensional point to set one prediction mode from two or more prediction modes for calculating a predicted value of the first position information of the first three-dimensional point, calculates the predicted value of the set prediction mode, calculates a prediction residual which is the difference between the first position information and the calculated predicted value, and generates a first bitstream including the set prediction mode and the prediction residual, and in the setting, sets the prediction mode based on the depth of the hierarchical structure of the first three-dimensional point.

[0012] A three-dimensional data decoding method according to one embodiment of the present disclosure is a three-dimensional data decoding method for decoding a plurality of three-dimensional points having a hierarchical structure, the method comprising: obtaining a first bitstream including an encoded prediction mode of a first three-dimensional point among the plurality of three-dimensional points and an encoded prediction residual; decoding a prediction mode value indicating the encoded prediction mode and the encoded prediction residual; calculating a prediction value of the prediction mode indicated by the prediction mode value obtained by decoding; and calculating first position information of the first three-dimensional point by adding the prediction value and the prediction residual obtained by decoding; the encoded prediction mode included in the first bitstream is a prediction mode set based on the depth of the hierarchical structure of the first three-dimensional point. [Effects of the Invention]

[0013] The present disclosure can provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing a configuration of a three-dimensional data encoding / decoding system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of point cloud data according to the first embodiment. [Figure 3]FIG. 3 is a diagram showing an example of the structure of a data file in which point cloud data information according to the first embodiment is described. [Figure 4] FIG. 4 is a diagram showing types of point cloud data according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing a configuration of a first encoding unit according to the first embodiment. [Figure 6] FIG. 6 is a block diagram of a first encoding unit according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating a configuration of a first decoding unit according to the first embodiment. [Figure 8] FIG. 8 is a block diagram of a first decoding unit according to the first embodiment. [Figure 9] FIG. 9 is a block diagram of a three-dimensional data encoding device according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of location information according to the first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of an octree representation of position information according to the first embodiment. [Figure 12] FIG. 12 is a block diagram of a three-dimensional data decoding device according to the first embodiment. [Figure 13] FIG. 13 is a block diagram of the attribute information encoding unit according to the first embodiment. [Figure 14] FIG. 14 is a block diagram of the attribute information decoding unit according to the first embodiment. [Figure 15] FIG. 15 is a block diagram showing a configuration of an attribute information encoding unit according to the first embodiment. [Figure 16] FIG. 16 is a block diagram of the attribute information encoding unit according to the first embodiment. [Figure 17] FIG. 17 is a block diagram showing a configuration of an attribute information decoding unit according to the first embodiment. [Figure 18] FIG. 18 is a block diagram of the attribute information decoding unit according to the first embodiment. [Figure 19] FIG. 19 is a diagram showing a configuration of a second encoding unit according to the first embodiment. [Figure 20]FIG. 20 is a block diagram of a second encoding unit according to the first embodiment. [Figure 21] FIG. 21 is a diagram illustrating a configuration of a second decoding unit according to the first embodiment. [Figure 22] FIG. 22 is a block diagram of a second decoding unit according to the first embodiment. [Figure 23] FIG. 23 is a diagram illustrating a protocol stack related to PCC encoded data according to the first embodiment. [Figure 24] FIG. 24 is a diagram illustrating the configurations of an encoding unit and a multiplexing unit according to the second embodiment. [Figure 25] FIG. 25 is a diagram illustrating an example of a structure of coded data according to the second embodiment. [Figure 26] FIG. 26 is a diagram illustrating an example of the structure of coded data and NAL units according to the second embodiment. [Figure 27] FIG. 27 is a diagram illustrating an example of the semantics of pcc_nal_unit_type according to the second embodiment. [Figure 28] FIG. 28 is a diagram showing an example of the transmission order of NAL units according to the second embodiment. [Figure 29] FIG. 29 is a flowchart of processing by the three-dimensional data encoding device according to the second embodiment. [Figure 30] FIG. 30 is a flowchart of processing by the three-dimensional data decoding device according to the second embodiment. [Figure 31] FIG. 31 is a flowchart of the multiplexing process according to the second embodiment. [Figure 32] FIG. 32 is a flowchart of the demultiplexing process according to the second embodiment. [Figure 33] FIG. 33 is a diagram illustrating a configuration of a distribution system according to the third embodiment. [Figure 34] FIG. 34 is a diagram illustrating an example of a configuration of a bit stream of an encoded 3D map according to the third embodiment. [Figure 35] FIG. 35 is a diagram for explaining the effect of improving the coding efficiency according to the third embodiment. [Figure 36]FIG. 36 is a flowchart of processing by the server according to the third embodiment. [Figure 37] FIG. 37 is a flowchart of processing by a client according to the third embodiment. [Figure 38] FIG. 38 is a diagram illustrating an example of the syntax of a submap according to the third embodiment. [Figure 39] FIG. 39 is a diagram schematically illustrating the coding type switching process according to the third embodiment. [Figure 40] FIG. 40 is a diagram illustrating an example of the syntax of a submap according to the third embodiment. [Figure 41] FIG. 41 is a flowchart of three-dimensional data encoding processing according to the third embodiment. [Figure 42] FIG. 42 is a flowchart of three-dimensional data decoding processing according to the third embodiment. [Figure 43] FIG. 43 is a diagram schematically illustrating the operation of a modified example of the coding type switching process according to the third embodiment. [Figure 44] FIG. 44 is a diagram schematically illustrating the operation of a modified example of the coding type switching process according to the third embodiment. [Figure 45] FIG. 45 is a diagram schematically illustrating the operation of a modified example of the coding type switching process according to the third embodiment. [Figure 46] FIG. 46 is a diagram illustrating the operation of a variation of the differential value calculation process according to the third embodiment. [Figure 47] FIG. 47 is a diagram illustrating the operation of a variation of the differential value calculation process according to the third embodiment. [Figure 48] FIG. 48 is a diagram illustrating the operation of a variation of the differential value calculation process according to the third embodiment. [Figure 49] FIG. 49 is a diagram illustrating the operation of a variation of the differential value calculation process according to the third embodiment. [Figure 50] FIG. 50 is a diagram illustrating an example of the syntax of a volume according to the third embodiment. [Figure 51]FIG. 51 is a diagram showing an example of three-dimensional points according to the fourth embodiment. [Figure 52] FIG. 52 is a diagram illustrating an example of setting LoD according to the fourth embodiment. [Figure 53] FIG. 53 is a diagram illustrating an example of threshold values ​​used for setting LoD according to the fourth embodiment. [Figure 54] FIG. 54 is a diagram showing an example of attribute information used for a predicted value according to the fourth embodiment. [Figure 55] FIG. 55 is a diagram illustrating an example of an exponential-Golomb code according to the fourth embodiment. [Figure 56] FIG. 56 is a diagram illustrating processing for exponential-Golomb coding according to the fourth embodiment. [Figure 57] FIG. 57 is a diagram illustrating an example of the syntax of an attribute header according to the fourth embodiment. [Figure 58] FIG. 58 is a diagram illustrating an example of the syntax of attribute data according to the fourth embodiment. [Figure 59] FIG. 59 is a flowchart of three-dimensional data encoding processing according to the fourth embodiment. [Figure 60] FIG. 60 is a flowchart of the attribute information encoding process according to the fourth embodiment. [Figure 61] FIG. 61 is a diagram illustrating processing for exponential-Golomb coding according to the fourth embodiment. [Figure 62] FIG. 62 is a diagram showing an example of a reverse lookup table showing the relationship between the remaining codes and their values ​​according to the fourth embodiment. [Figure 63] FIG. 63 is a flowchart of three-dimensional data decoding processing according to the fourth embodiment. [Figure 64] FIG. 64 is a flowchart of the attribute information decoding process according to the fourth embodiment. [Figure 65] FIG. 65 is a block diagram of a three-dimensional data encoding device according to the fourth embodiment. [Figure 66] FIG. 66 is a block diagram of a three-dimensional data decoding device according to the fourth embodiment. [Figure 67]FIG. 67 is a diagram showing a first example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. In FIG. [Figure 68] FIG. 68 is a diagram showing an example of attribute information used for a predicted value according to the fifth embodiment. [Figure 69] FIG. 69 is a diagram showing a second example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. In FIG. [Figure 70] FIG. 70 is a diagram showing a third example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. In FIG. [Figure 71] FIG. 71 is a diagram showing a fourth example of a table indicating predicted values ​​calculated in each prediction mode according to the fifth embodiment. [Figure 72] FIG. 72 is a diagram showing a fifth example of a table indicating predicted values ​​calculated in each prediction mode according to the fifth embodiment. [Figure 73] FIG. 73 is a diagram showing a sixth example of a table indicating predicted values ​​calculated in each prediction mode according to the fifth embodiment. [Figure 74] FIG. 74 is a diagram showing a seventh example of a table indicating predicted values ​​calculated in each prediction mode according to the fifth embodiment. [Figure 75] FIG. 75 is a diagram showing an eighth example of a table indicating predicted values ​​calculated in each prediction mode according to the fifth embodiment. [Figure 76] FIG. 76 is a diagram showing a first example of a binarization table when binarizing and encoding prediction mode values ​​according to the fifth embodiment. [Figure 77] FIG. 77 is a diagram showing a second example of a binarization table when binarizing and encoding prediction mode values ​​according to the fifth embodiment. [Figure 78] FIG. 78 is a diagram showing a third example of a binarization table when binarizing and encoding prediction mode values ​​according to the fifth embodiment. [Figure 79] FIG. 79 is a diagram illustrating an example of encoding binary data of a binarization table when binarizing and encoding a prediction mode according to the fifth embodiment. [Figure 80] FIG. 80 is a flowchart showing an example of encoding of prediction mode values ​​according to the fifth embodiment. [Figure 81] FIG. 81 is a flowchart showing an example of decoding of prediction mode values ​​according to the fifth embodiment. [Figure 82] FIG. 82 is a diagram showing another example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. In FIG. [Figure 83] FIG. 83 is a diagram illustrating an example of encoding binary data of a binarization table when binarizing and encoding a prediction mode according to the fifth embodiment. In FIG. [Figure 84] FIG. 84 is a flowchart showing another example of encoding of prediction mode values ​​according to the fifth embodiment. [Figure 85] FIG. 85 is a flowchart showing another example of decoding of prediction mode values ​​according to the fifth embodiment. [Figure 86] FIG. 86 is a flowchart showing an example of processing for determining whether or not to fix the prediction mode value in accordance with condition A during encoding according to the fifth embodiment. [Figure 87] FIG. 87 is a flowchart showing an example of processing for determining whether to set a prediction mode value to a fixed value or decode the prediction mode value in accordance with condition A during decoding according to the fifth embodiment. [Figure 88] FIG. 88 is a diagram for explaining the maximum absolute difference value maxdiff according to the fifth embodiment. [Figure 89] FIG. 89 is a diagram illustrating an example of syntax according to the fifth embodiment. [Figure 90] FIG. 90 is a diagram illustrating an example of syntax according to the fifth embodiment. [Figure 91] FIG. 91 is a flowchart of three-dimensional data encoding processing by the three-dimensional data encoding device according to the fifth embodiment. [Figure 92] FIG. 92 is a flowchart of the attribute information encoding process according to the fifth embodiment. [Figure 93]FIG. 93 is a flowchart of a calculation process of a predicted value in the three-dimensional data encoding device according to the fifth embodiment. [Figure 94] FIG. 94 is a flowchart of a prediction mode selection process according to the fifth embodiment. [Figure 95] FIG. 95 is a flowchart showing the process of selecting a prediction mode with the minimum cost according to the fifth embodiment. [Figure 96] FIG. 96 is a flowchart of three-dimensional data decoding processing by the three-dimensional data decoding device according to the fifth embodiment. [Figure 97] FIG. 97 is a flowchart of the attribute information decoding process according to the fifth embodiment. [Figure 98] FIG. 98 is a flowchart of the process of calculating a predicted value in the three-dimensional data decoding device according to the fifth embodiment. [Figure 99] FIG. 99 is a flowchart of the prediction mode decoding process according to the fifth embodiment. [Figure 100] FIG. 100 is a block diagram showing the configuration of an attribute information encoding unit included in the three-dimensional data encoding device according to the fifth embodiment. [Figure 101] FIG. 101 is a block diagram showing the configuration of an attribute information decoding unit included in the three-dimensional data decoding device according to the fifth embodiment. [Figure 102] FIG. 102 is a flowchart showing a prediction mode determination process executed by the three-dimensional data encoding device according to the sixth embodiment. [Figure 103] FIG. 103 is a diagram showing an example of syntax for the process of determining a prediction mode executed by the three-dimensional data encoding device according to the sixth embodiment. [Figure 104] FIG. 104 is a flowchart showing a prediction mode determination process executed by the three-dimensional data decoding device according to the sixth embodiment. [Figure 105] FIG. 105 is a diagram showing an example of the syntax of attribute data in a case where a prediction mode fixing flag according to the sixth embodiment is provided for each 3D point. [Figure 106]FIG. 106 is a diagram showing an example of the syntax of attribute data in a case where a prediction mode fixing flag according to the sixth embodiment is provided for each layer of LoD. [Figure 107] FIG. 107 is a flowchart showing an example of the prediction mode encoding process of the three-dimensional data encoding device according to the sixth embodiment. [Figure 108] FIG. 108 is a flowchart showing an example of the prediction mode decoding process of the three-dimensional data decoding device according to the sixth embodiment. [Figure 109] FIG. 109 is a flowchart of three-dimensional data encoding processing by the three-dimensional data encoding device according to the sixth embodiment. [Figure 110] FIG. 110 is a flowchart of the attribute information encoding process shown in FIG. [Figure 111] FIG. 111 is a flowchart of the calculation process of the predicted value shown in FIG. [Figure 112] FIG. 112 is a flowchart of the prediction mode selection process shown in FIG. [Figure 113] FIG. 113 is a flowchart showing a specific example of the prediction mode selection process shown in FIG. [Figure 114] FIG. 114 is a flowchart of three-dimensional data decoding processing by the three-dimensional data decoding device according to the sixth embodiment. [Figure 115] FIG. 115 is a flowchart of the calculation process of the predicted value shown in FIG. [Figure 116] FIG. 116 is a flowchart showing the details of the calculation process of the predicted value shown in FIG. [Figure 117] FIG. 117 is a flowchart of the process of calculating the prediction mode and quantization value shown in FIG. [Figure 118] FIG. 118 is a flowchart of processing when the three-dimensional data encoding device according to the sixth embodiment does not fix the prediction mode. [Figure 119] FIG. 119 is a flowchart of processing when the three-dimensional data decoding device according to the sixth embodiment does not fix the prediction mode. [Figure 120] FIG. 120 is a diagram showing another example of the syntax of the attribute data according to the sixth embodiment. [Figure 121] FIG. 121 is a flowchart showing an example of the prediction mode encoding process of the three-dimensional data encoding device according to the sixth embodiment. [Figure 122] FIG. 122 is a flowchart showing an example of a prediction mode decoding process of the three-dimensional data decoding device according to the sixth embodiment. [Figure 123] FIG. 123 is a flowchart showing another example of the calculation process of the predicted value shown in FIG. [Figure 124] FIG. 124 is a flowchart of the prediction mode selection process shown in FIG. [Figure 125] FIG. 125 is a flowchart showing a specific example of the prediction mode selection process shown in FIG. [Figure 126] FIG. 126 is a flowchart showing another example of the process of calculating prediction modes and quantization values ​​shown in FIG. [Figure 127] FIG. 127 is a diagram showing an example of attribute information used for a predicted value. [Figure 128] FIG. 128 is a diagram illustrating an example of a reference relationship according to the seventh embodiment. [Figure 129] FIG. 129 is a diagram showing an example of calculation of the QW according to the seventh embodiment. [Figure 130] FIG. 130 is a diagram showing an example of calculation of a prediction residual according to the seventh embodiment. [Figure 131] FIG. 131 is a diagram showing an example of a prediction tree used in a three-dimensional data coding method according to the eighth embodiment. [Figure 132] FIG. 132 is a flowchart showing an example of a three-dimensional data encoding method according to the eighth embodiment. [Figure 133] FIG. 133 is a flowchart showing an example of a three-dimensional data decoding method according to the eighth embodiment. [Figure 134] FIG. 134 is a diagram illustrating a method for generating a prediction tree according to the eighth embodiment. [Figure 135]FIG. 135 is a diagram illustrating a first example of a prediction mode according to the eighth embodiment. [Figure 136] FIG. 136 is a diagram showing a second example of a table indicating predicted values ​​calculated in each prediction mode according to the eighth embodiment. [Figure 137] FIG. 137 is a diagram showing a specific example of a second example of a table indicating predicted values ​​calculated in each prediction mode according to the eighth embodiment. [Figure 138] FIG. 138 is a diagram showing a third example of a table indicating predicted values ​​calculated in each prediction mode according to the eighth embodiment. [Figure 139] FIG. 139 is a diagram showing a fourth example of a table indicating predicted values ​​calculated in each prediction mode according to the eighth embodiment. [Figure 140] FIG. 140 is a diagram showing a fifth example of a table indicating predicted values ​​calculated in each prediction mode according to the eighth embodiment. [Figure 141] FIG. 141 is a diagram showing a sixth example of a table indicating predicted values ​​calculated in each prediction mode according to the eighth embodiment. [Figure 142] FIG. 142 is a diagram showing a seventh example of a table indicating predicted values ​​calculated in each prediction mode according to the eighth embodiment. [Figure 143] FIG. 143 is a diagram showing a first example of a binarization table when prediction mode values ​​according to the eighth embodiment are binarized and coded. [Figure 144] FIG. 144 is a diagram showing a second example of a binarization table when binarizing and encoding prediction mode values ​​according to the eighth embodiment. [Figure 145] FIG. 145 is a diagram showing a third example of a binarization table when binarizing and encoding prediction mode values ​​according to the eighth embodiment. [Figure 146] FIG. 146 is a diagram illustrating an example of encoding binary data of a binarization table when binarizing and encoding a prediction mode according to the eighth embodiment. In FIG. [Figure 147]FIG. 147 is a flowchart showing an example of encoding of prediction mode values ​​according to the eighth embodiment. [Figure 148] FIG. 148 is a flowchart showing an example of decoding of prediction mode values ​​according to the eighth embodiment. [Figure 149] FIG. 149 is a diagram showing another example of a table indicating predicted values ​​calculated in each prediction mode according to the eighth embodiment. [Figure 150] FIG. 150 is a diagram illustrating an example of encoding binary data of a binarization table when binarizing and encoding a prediction mode according to the eighth embodiment. [Figure 151] FIG. 151 is a flowchart showing another example of encoding of prediction mode values ​​according to the eighth embodiment. [Figure 152] FIG. 152 is a flowchart showing another example of decoding of prediction mode values ​​according to the eighth embodiment. [Figure 153] FIG. 153 is a flowchart showing an example of processing for determining whether or not to fix the prediction mode value in accordance with the condition A during encoding according to the eighth embodiment. [Fig. 154] FIG. 154 is a flowchart showing an example of processing for determining whether to set a prediction mode value to a fixed value or decode the prediction mode value in accordance with condition A during decoding according to the eighth embodiment. [Figure 155] FIG. 155 is a diagram illustrating an example of the syntax of the header of the location information according to the eighth embodiment. [Figure 156] FIG. 156 is a diagram illustrating an example of the syntax of the location information according to the eighth embodiment. [Figure 157] FIG. 157 is a diagram illustrating another example of the syntax of the location information according to the eighth embodiment. [Figure 158] FIG. 158 is a diagram showing an example of a prediction tree used in a three-dimensional data coding method according to the ninth embodiment. [Figure 159] FIG. 159 is a diagram illustrating another example of the syntax of the location information according to the ninth embodiment. [Figure 160]FIG. 160 is a diagram showing an example of a structure of a prediction tree commonly used for encoding position information and attribute information according to the ninth embodiment. [Figure 161] FIG. 161 is a flowchart showing an example of a three-dimensional data encoding method according to a modification of the ninth embodiment. [Figure 162] FIG. 162 is a flowchart showing an example of a three-dimensional data decoding method according to a modification of the ninth embodiment. [Figure 163] FIG. 163 is a diagram illustrating an example of the syntax of the header of the attribute information according to the ninth embodiment. [Fig. 164] FIG. 164 is a diagram showing another example of the syntax of the attribute information according to the ninth embodiment. [Figure 165] FIG. 165 is a diagram showing an example of the syntax of the position information and attribute information according to the ninth embodiment. [Figure 166] FIG. 166 is a flowchart of the processing performed by the three-dimensional data encoding device according to the eighth and ninth embodiments. [Figure 167] FIG. 167 is a flowchart of the processing of the three-dimensional data decoding device according to the eighth and ninth embodiments. [Figure 168] FIG. 168 is a block diagram of a three-dimensional data creation device according to the tenth embodiment. [Figure 169] FIG. 169 is a flowchart of a three-dimensional data creation method according to the tenth embodiment. [Figure 170] FIG. 170 is a diagram showing the configuration of a system according to the tenth embodiment. [Figure 171] FIG. 171 is a block diagram of a client device according to the tenth embodiment. [Figure 172] FIG. 172 is a block diagram of a server according to the tenth embodiment. [Fig. 173] FIG. 173 is a flowchart of three-dimensional data creation processing by a client device according to the tenth embodiment. [Fig. 174] FIG. 174 is a flowchart of a sensor information transmission process by a client device according to the tenth embodiment. [Figure 175] FIG. 175 is a flowchart of three-dimensional data creation processing by the server according to the tenth embodiment. [Figure 176] FIG. 176 is a flowchart of a three-dimensional map transmission process performed by the server according to the tenth embodiment. [Figure 177] FIG. 177 is a diagram showing the configuration of a modified example of the system according to the tenth embodiment. [Figure 178] FIG. 178 is a diagram showing the configurations of a server and a client device according to the tenth embodiment. [Figure 179] FIG. 179 is a diagram showing the configurations of a server and a client device according to the tenth embodiment. [Figure 180] FIG. 180 is a flowchart of processing by a client device according to the tenth embodiment. [Figure 181] FIG. 181 is a diagram illustrating a configuration of a sensor information collection system according to the tenth embodiment. [Figure 182] FIG. 182 is a diagram illustrating an example of a system according to the tenth embodiment. [Figure 183] FIG. 183 is a diagram showing a modified example of the system according to the tenth embodiment. [Figure 184] FIG. 184 is a flowchart showing an example of application processing according to the tenth embodiment. [Figure 185] FIG. 185 is a diagram showing the sensor ranges of various sensors according to the tenth embodiment. [Figure 186] Figure 186 is a diagram showing an example of the configuration of an autonomous driving system according to embodiment 10. [Figure 187] FIG. 187 is a diagram showing an example of the structure of a bitstream according to the tenth embodiment. [Figure 188] FIG. 188 is a flowchart of the point group selection process according to the tenth embodiment. [Figure 189] FIG. 189 is a diagram showing an example of a screen for the point group selection process according to the tenth embodiment. [Figure 190] FIG. 190 is a diagram showing an example of a screen for the point group selection process according to the tenth embodiment. [Figure 191] FIG. 191 is a diagram showing an example of a screen for the point group selection process according to the tenth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] A three-dimensional data encoding method according to one embodiment of the present disclosure is a three-dimensional data encoding method for encoding a plurality of three-dimensional points having a hierarchical structure, which uses second position information of one or more second three-dimensional points surrounding a first three-dimensional point to set one prediction mode from two or more prediction modes for calculating a predicted value of the first position information of the first three-dimensional point, calculates the predicted value of the set prediction mode, calculates a prediction residual which is the difference between the first position information and the calculated predicted value, and generates a first bitstream including the set prediction mode and the prediction residual, and in the setting, sets the prediction mode based on the depth of the hierarchical structure of the first three-dimensional point.

[0016] This allows the position information to be encoded using the predicted value of one prediction mode out of two or more prediction modes, which is set based on the depth of the hierarchical structure, thereby improving the encoding efficiency of the position information.

[0017] Furthermore, the setting may include setting a prediction mode value that is equal to or less than the value of the depth of the hierarchical structure of the first 3D point, and the prediction mode value may indicate the prediction mode.

[0018] Furthermore, the first bitstream may further include a prediction mode number indicating the number of the two or more prediction modes.

[0019] In addition, in the generation, a prediction mode value indicating the set prediction mode may be encoded using the number of prediction modes, and the first bitstream may include the encoded prediction mode value as the set prediction mode.

[0020] In addition, in the generation, the prediction mode value may be coded using a truncated unary code in which the number of prediction modes is set to a maximum value.

[0021] This makes it possible to reduce the amount of coding for the prediction mode values.

[0022] Furthermore, each of the first position information and the second position information may include three elements, and in the setting, a common prediction mode may be set for the three elements as the one prediction mode for calculating predicted values ​​for each of the three elements included in the first position information.

[0023] This makes it possible to reduce the amount of coding for the prediction mode values.

[0024] Furthermore, each of the first position information and the second position information may include three elements, and in the setting, an independent prediction mode may be set for each of the three elements as the one prediction mode for calculating a predicted value for each of the three elements included in the first position information.

[0025] Therefore, the three-dimensional data decoding device can decode each element independently.

[0026] Furthermore, each of the first position information and the second position information may include three elements, and in the setting, as the one prediction mode for calculating predicted values ​​of each of the three elements included in the first position information, a common prediction mode may be set for two of the three elements, and a prediction mode independent of the two elements may be set for the remaining element.

[0027] This allows the amount of coding of the prediction mode values ​​for the two elements to be reduced, and the three-dimensional data decoding device can independently decode the remaining element.

[0028] Furthermore, in the generation, if the number of prediction modes is 1, a prediction mode value indicating the prediction mode may not be coded, and a second bitstream not including the prediction mode value may be generated.

[0029] This makes it possible to reduce the amount of code in the bitstream.

[0030] In addition, in the generation, if a prediction mode is set in which the predicted value calculated in the calculation is 0, positive / negative information indicating whether the prediction residual is positive or negative may not be encoded, and a third bitstream may be generated that does not include the positive / negative information.

[0031] This makes it possible to reduce the amount of code in the bitstream.

[0032] A three-dimensional data decoding method according to one embodiment of the present disclosure is a three-dimensional data decoding method for decoding a plurality of three-dimensional points having a hierarchical structure, the method comprising: obtaining a first bitstream including an encoded prediction mode of a first three-dimensional point among the plurality of three-dimensional points and an encoded prediction residual; decoding a prediction mode value indicating the encoded prediction mode and the encoded prediction residual; calculating a prediction value of the prediction mode indicated by the prediction mode value obtained by decoding; and calculating first position information of the first three-dimensional point by adding the prediction value and the prediction residual obtained by decoding; the encoded prediction mode included in the first bitstream is a prediction mode set based on the depth of the hierarchical structure of the first three-dimensional point.

[0033] This makes it possible to appropriately decode the position information that has been coded using a predicted value of one prediction mode, which is set based on the depth of the hierarchical structure, out of two or more prediction modes.

[0034] Furthermore, a prediction mode value indicating an encoded prediction mode included in the first bitstream may be equal to or less than a value of the depth of the hierarchical structure of the first 3D point.

[0035] The first bitstream may also include a prediction mode number indicating the number of the two or more prediction modes.

[0036] Furthermore, in the decoding, the coded prediction mode values ​​may be decoded using a truncated unary code in which the number of prediction modes is set to a maximum value.

[0037] Furthermore, each of the first position information and the second position information of one or more second three-dimensional points surrounding the first three-dimensional point may include three elements, and the prediction mode may be used to calculate a predicted value of each of the three elements included in the first position information, and may be set commonly for the three elements.

[0038] Furthermore, each of the first position information and the second position information of one or more second three-dimensional points surrounding the first three-dimensional point may include three elements, and the prediction mode may be used to calculate a predicted value of each of the three elements included in the first position information, and may be set independently for each of the three elements.

[0039] Furthermore, each of the first position information and the second position information of one or more second three-dimensional points surrounding the first three-dimensional point may include three elements, and the prediction mode may be used to calculate a predicted value of each of the three elements included in the first position information, and may be set commonly for two of the three elements, and may be set independently of the two elements for the remaining element.

[0040] Furthermore, when a second bitstream not including the prediction mode value is acquired in the acquisition, the calculation of the predicted value may include calculating a predicted value of a specific prediction mode.

[0041] Furthermore, if a third bitstream is acquired during the acquisition that does not include positive / negative information indicating whether the prediction residual is positive or negative, the prediction residual may be treated as 0 or a positive number when calculating the first position information.

[0042] Furthermore, a three-dimensional data encoding device according to one embodiment of the present disclosure is a three-dimensional data encoding device for encoding a plurality of three-dimensional points, comprising a processor and a memory, wherein the processor uses the memory to encode a plurality of three-dimensional points having a hierarchical structure. The processor is a three-dimensional data encoding method for encoding a plurality of three-dimensional points having a hierarchical structure, wherein the processor uses second position information of one or more second three-dimensional points surrounding a first three-dimensional point to set one prediction mode out of two or more prediction modes for calculating a predicted value of first position information of the first three-dimensional point, calculates a predicted value of the set prediction mode, calculates a prediction residual which is the difference between the first position information and the calculated predicted value, and generates a first bitstream including the set prediction mode and the prediction residual, wherein the setting sets the prediction mode based on the depth of the hierarchical structure of the first three-dimensional point.

[0043] This allows the position information to be encoded using the predicted value of one prediction mode out of two or more prediction modes, which is set based on the depth of the hierarchical structure, thereby improving the encoding efficiency of the position information.

[0044] Furthermore, a three-dimensional data decoding device according to one aspect of the present disclosure is a three-dimensional data decoding device that decodes a plurality of three-dimensional points, and includes a processor and a memory. The processor uses the memory to provide a three-dimensional data decoding method for decoding a plurality of three-dimensional points having a hierarchical structure, the method comprising: obtaining a first bitstream including an encoded prediction mode of a first three-dimensional point among the plurality of three-dimensional points and an encoded prediction residual; decoding a prediction mode value indicating the encoded prediction mode and the encoded prediction residual; calculating a prediction value of the prediction mode indicated by the prediction mode value obtained by decoding; and calculating first position information of the first three-dimensional point by adding the prediction value and the prediction residual obtained by decoding; the encoded prediction mode included in the first bitstream is a prediction mode set based on the depth of the hierarchical structure of the first three-dimensional point.

[0045] This makes it possible to appropriately decode the position information that has been coded using a predicted value of one prediction mode, which is set based on the depth of the hierarchical structure, out of two or more prediction modes.

[0046] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0047] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in an independent claim that represents a superordinate concept will be described as optional components.

[0048] (Embodiment 1) When using encoded point cloud data in an actual device or service, it is desirable to transmit and receive the information required for the application in order to reduce network bandwidth. However, until now, such a function has not existed in the encoding structure of 3D data, and no encoding method for this purpose has existed.

[0049] In this embodiment, we will describe a three-dimensional data encoding method and a three-dimensional data encoding device that provide the function of transmitting and receiving information required for the purpose in encoded data of a three-dimensional point cloud, as well as a three-dimensional data decoding method and a three-dimensional data decoding device that decodes the encoded data, a three-dimensional data multiplexing method that multiplexes the encoded data, and a three-dimensional data transmission method that transmits the encoded data.

[0050] In particular, two encoding methods (encoding schemes) are currently being considered as encoding methods (encoding systems) for point cloud data; however, the structure of the encoded data and the method for storing the encoded data in a system format have not been defined, and as things stand, there is a problem that MUX processing (multiplexing) in the encoding unit, or transmission or storage, is not possible.

[0051] Furthermore, there has been no method to date that supports a format in which two codecs, a first encoding method and a second encoding method, are mixed, such as PCC (Point Cloud Compression).

[0052] In this embodiment, a description will be given of the structure of PCC encoded data in which two codecs, a first encoding method and a second encoding method, are mixed, and a method of storing the encoded data in a system format.

[0053] First, the configuration of a three-dimensional data (point cloud data) encoding / decoding system according to this embodiment will be described. Fig. 1 is a diagram showing an example of the configuration of a three-dimensional data encoding / decoding system according to this embodiment. As shown in Fig. 1, the three-dimensional data encoding / decoding system includes a three-dimensional data encoding system 4601, a three-dimensional data decoding system 4602, a sensor terminal 4603, and an external connection unit 4604.

[0054] The three-dimensional data encoding system 4601 generates encoded data or multiplexed data by encoding point cloud data, which is three-dimensional data. Note that the three-dimensional data encoding system 4601 may be a three-dimensional data encoding device realized by a single device, or may be a system realized by multiple devices. Furthermore, the three-dimensional data encoding device may include some of the multiple processing units included in the three-dimensional data encoding system 4601.

[0055] The three-dimensional data encoding system 4601 includes a point cloud data generation system 4611, a presentation unit 4612, an encoding unit 4613, a multiplexing unit 4614, an input / output unit 4615, and a control unit 4616. The point cloud data generation system 4611 includes a sensor information acquisition unit 4617 and a point cloud data generation unit 4618.

[0056] The sensor information acquisition unit 4617 acquires sensor information from the sensor terminal 4603 and outputs the sensor information to the point cloud data generation unit 4618. The point cloud data generation unit 4618 generates point cloud data from the sensor information and outputs the point cloud data to the encoding unit 4613.

[0057] The presentation unit 4612 presents the sensor information or point cloud data to the user. For example, the presentation unit 4612 displays information or an image based on the sensor information or point cloud data.

[0058] The encoding unit 4613 encodes (compresses) the point cloud data, and outputs the obtained encoded data, control information obtained in the encoding process, and other additional information to the multiplexing unit 4614. The additional information includes, for example, sensor information.

[0059] The multiplexing unit 4614 generates multiplexed data by multiplexing the coded data input from the coding unit 4613, the control information, and the additional information. The format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.

[0060] The input / output unit 4615 (e.g., a communication unit or an interface) outputs the multiplexed data to the outside. Alternatively, the multiplexed data is stored in a storage unit such as an internal memory. The control unit 4616 (or application execution unit) controls each processing unit. In other words, the control unit 4616 controls encoding, multiplexing, etc.

[0061] The sensor information may be input to the encoding unit 4613 or the multiplexing unit 4614. The input / output unit 4615 may output the point cloud data or the encoded data directly to the outside.

[0062] The transmission signal (multiplexed data) output from the three-dimensional data encoding system 4601 is input to the three-dimensional data decoding system 4602 via the external connection unit 4604 .

[0063] The three-dimensional data decoding system 4602 generates point cloud data, which is three-dimensional data, by decoding the encoded data or multiplexed data. Note that the three-dimensional data decoding system 4602 may be a three-dimensional data decoding device realized by a single device, or may be a system realized by multiple devices. Furthermore, the three-dimensional data decoding device may include some of the multiple processing units included in the three-dimensional data decoding system 4602.

[0064] The three-dimensional data decoding system 4602 includes a sensor information acquisition unit 4621 , an input / output unit 4622 , a demultiplexing unit 4623 , a decoding unit 4624 , a presentation unit 4625 , a user interface 4626 , and a control unit 4627 .

[0065] The sensor information acquisition unit 4621 acquires sensor information from the sensor terminal 4603 .

[0066] The input / output unit 4622 acquires the transmission signal, decodes the multiplexed data (file format or packets) from the transmission signal, and outputs the multiplexed data to the demultiplexer 4623.

[0067] The demultiplexing unit 4623 obtains the coded data, control information, and additional information from the multiplexed data, and outputs the coded data, control information, and additional information to the decoding unit 4624.

[0068] The decoding unit 4624 reconstructs the point cloud data by decoding the encoded data.

[0069] The presentation unit 4625 presents the point cloud data to the user. For example, the presentation unit 4625 displays information or images based on the point cloud data. The user interface 4626 acquires instructions based on user operations. The control unit 4627 (or the application execution unit) controls each processing unit. In other words, the control unit 4627 controls demultiplexing, decoding, presentation, etc.

[0070] The input / output unit 4622 may acquire point cloud data or encoded data directly from the outside. The presentation unit 4625 may acquire additional information such as sensor information and present information based on the additional information. The presentation unit 4625 may perform presentation based on a user instruction acquired by the user interface 4626.

[0071] The sensor terminal 4603 generates sensor information, which is information obtained by a sensor. The sensor terminal 4603 is a terminal equipped with a sensor or a camera, and examples thereof include a moving body such as an automobile, a flying object such as an airplane, a mobile terminal, or a camera.

[0072] The sensor information that can be acquired by the sensor terminal 4603 includes, for example, (1) the distance between the sensor terminal 4603 and an object, or the reflectance of the object, obtained from a LIDAR, millimeter-wave radar, or infrared sensor, and (2) the distance between a camera and an object, or the reflectance of the object, obtained from multiple monocular camera images or stereo camera images. The sensor information may also include the attitude, direction, gyro (angular velocity), position (GPS information or altitude), speed, acceleration, etc. of the sensor. The sensor information may also include temperature, air pressure, humidity, magnetism, etc.

[0073] The external connection unit 4604 is realized by an integrated circuit (LSI or IC), an external storage unit, communication with a cloud server via the Internet, broadcasting, or the like.

[0074] Next, point cloud data will be described. Fig. 2 is a diagram showing the configuration of point cloud data. Fig. 3 is a diagram showing an example of the configuration of a data file in which information about point cloud data is written.

[0075] Point cloud data includes data on multiple points. The data on each point includes location information (three-dimensional coordinates) and attribute information for that location information. A collection of multiple points is called a point cloud. For example, a point cloud can represent the three-dimensional shape of an object.

[0076] Position information such as three-dimensional coordinates is sometimes called geometry. Data for each point may also include attribute information of multiple attribute types. Attribute types include, for example, color or reflectance.

[0077] One piece of attribute information may be associated with one piece of location information, or multiple pieces of attribute information with different attribute types may be associated with one piece of location information, or multiple pieces of attribute information of the same attribute type may be associated with one piece of location information.

[0078] The configuration example of the data file shown in FIG. 3 is an example in which there is a one-to-one correspondence between position information and attribute information, and shows the position information and attribute information of N points that make up the point cloud data.

[0079] The position information is, for example, information on the three axes x, y, and z. The attribute information is, for example, RGB color information. A typical data file is a ply file.

[0080] Next, the types of point cloud data will be explained. Fig. 4 is a diagram showing the types of point cloud data. As shown in Fig. 4, point cloud data includes static objects and dynamic objects.

[0081] A static object is 3D point cloud data at any time (a certain time). A dynamic object is 3D point cloud data that changes over time. Hereinafter, 3D point cloud data at a certain time will be referred to as a PCC frame, or simply a frame.

[0082] The object may be a point cloud with a certain area restriction, such as ordinary video data, or a large-scale point cloud with no area restriction, such as map information.

[0083] Furthermore, there may be point cloud data of various densities, such as sparse point cloud data and dense point cloud data.

[0084] Each processing unit will be described in detail below. Sensor information is acquired by various methods, such as a distance sensor such as a LIDAR or a range finder, a stereo camera, or a combination of multiple monocular cameras. The point cloud data generation unit 4618 generates point cloud data based on the sensor information acquired by the sensor information acquisition unit 4617. The point cloud data generation unit 4618 generates position information as point cloud data, and adds attribute information for the position information to the position information.

[0085] The point cloud data generation unit 4618 may process the point cloud data when generating position information or adding attribute information. For example, the point cloud data generation unit 4618 may reduce the amount of data by deleting point clouds with overlapping positions. In addition, the point cloud data generation unit 4618 may convert (position shift, rotation, normalization, etc.) the position information or render the attribute information.

[0086] In FIG. 1, the point cloud data generation system 4611 is included in the three-dimensional data encoding system 4601, but it may be provided independently outside the three-dimensional data encoding system 4601.

[0087] The encoding unit 4613 generates encoded data by encoding the point cloud data based on a predetermined encoding method. There are two main types of encoding methods: the first is an encoding method that uses position information, and this encoding method will be referred to as the first encoding method hereinafter; and the second is an encoding method that uses a video codec, and this encoding method will be referred to as the second encoding method hereinafter.

[0088] The decoding unit 4624 decodes the coded data based on a predetermined coding method to decode the point group data.

[0089] The multiplexing unit 4614 generates multiplexed data by multiplexing the encoded data using an existing multiplexing method. The generated multiplexed data is transmitted or stored. In addition to the PCC encoded data, the multiplexing unit 4614 multiplexes other media such as video, audio, subtitles, applications, and files, or reference time information. The multiplexing unit 4614 may also multiplex attribute information related to sensor information or point cloud data.

[0090] Multiplexing methods or file formats include ISOBMFF, MPEG-DASH, which is an ISOBMFF-based transmission method, MMT, MPEG-2 TS Systems, and RMP.

[0091] The demultiplexer 4623 extracts PCC encoded data, other media, time information, and the like from the multiplexed data.

[0092] The input / output unit 4615 transmits the multiplexed data using a method suited to the transmission medium or storage medium, such as broadcasting or communication. The input / output unit 4615 may communicate with other devices via the Internet, or may communicate with a storage unit such as a cloud server.

[0093] The communication protocol used may be http, ftp, TCP, UDP, etc. A PULL type communication method or a PUSH type communication method may be used.

[0094] Either wired or wireless transmission may be used. For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), coaxial cable, etc. are used. For wireless transmission, wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), millimeter waves, etc. are used.

[0095] Furthermore, as a broadcasting system, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 is used.

[0096] Fig. 5 is a diagram showing the configuration of a first encoding unit 4630, which is an example of the encoding unit 4613 that performs encoding using the first encoding method. Fig. 6 is a block diagram of the first encoding unit 4630. The first encoding unit 4630 generates encoded data (encoded stream) by encoding point cloud data using the first encoding method. This first encoding unit 4630 includes a position information encoding unit 4631, an attribute information encoding unit 4632, an additional information encoding unit 4633, and a multiplexing unit 4634.

[0097] The first encoding unit 4630 is characterized in that it performs encoding taking into consideration a three-dimensional structure. The first encoding unit 4630 is also characterized in that the attribute information encoding unit 4632 performs encoding using information obtained from the position information encoding unit 4631. The first encoding method is also called GPCC (Geometry based PCC).

[0098] The point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData). The position information is input to a position information encoder 4631, the attribute information is input to an attribute information encoder 4632, and the additional information is input to an additional information encoder 4633.

[0099] The position information encoding unit 4631 encodes the position information to generate encoded position information (Compressed Geometry), which is encoded data. For example, the position information encoding unit 4631 encodes the position information using an N-ary tree structure such as an octree. Specifically, in an octree, the target space is divided into eight nodes (subspaces), and 8-bit information (occupancy code) indicating whether or not a point cloud is included in each node is generated. Furthermore, the node including the point cloud is further divided into eight nodes, and 8-bit information indicating whether or not a point cloud is included in each of the eight nodes is generated. This process is repeated until the number of point clouds included in a predetermined layer or node falls below a threshold.

[0100] The attribute information encoding unit 4632 generates encoded attribute information (Compressed Attribute) that is encoded data by encoding using the configuration information generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 determines a reference point (reference node) to be referenced when encoding a target point (target node) to be processed, based on the octree structure generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 references a peripheral node or adjacent node whose parent node in the octree is the same as that of the target node. Note that the method of determining the reference relationship is not limited to this.

[0101] Furthermore, the encoding process of the attribute information may include at least one of a quantization process, a prediction process, and an arithmetic coding process. In this case, the reference means using a reference node to calculate a predicted value of the attribute information, or using the state of the reference node (e.g., occupancy information indicating whether the reference node includes a point group) to determine an encoding parameter. For example, the encoding parameter is a quantization parameter in a quantization process, or a context in an arithmetic coding process.

[0102] The additional information encoding unit 4633 generates encoded additional information (Compressed MetaData) that is encoded data by encoding compressible data from the additional information.

[0103] The multiplexing unit 4634 multiplexes the encoding position information, the encoding attribute information, the encoding additional information, and other additional information to generate a compressed stream, which is encoded data. The generated compressed stream is output to a processing unit in a system layer (not shown).

[0104] Next, a first decoding unit 4640, which is an example of the decoding unit 4624 that performs decoding using the first encoding method, will be described. FIG. 7 is a diagram showing the configuration of the first decoding unit 4640. FIG. 8 is a block diagram of the first decoding unit 4640. The first decoding unit 4640 generates point cloud data by decoding, using the first encoding method, coded data (coded stream) coded using the first coding method. The first decoding unit 4640 includes a demultiplexing unit 4641, a position information decoding unit 4642, an attribute information decoding unit 4643, and an additional information decoding unit 4644.

[0105] A coded stream (compressed stream) that is coded data is input to the first decoding unit 4640 from a processing unit in a system layer (not shown).

[0106] The demultiplexer 4641 separates the encoded position information (Compressed Geometry), the encoded attribute information (Compressed Attribute), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0107] The position information decoding unit 4642 generates position information by decoding the encoded position information. For example, the position information decoding unit 4642 restores the position information of a point group represented by three-dimensional coordinates from the encoded position information represented by an N-ary tree structure such as an octree.

[0108] The attribute information decoding unit 4643 decodes the encoded attribute information based on the configuration information generated by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 determines a reference point (reference node) to be referenced in decoding the target point (target node) to be processed based on the octree structure obtained by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 references a peripheral node or adjacent node whose parent node in the octree is the same as that of the target node. Note that the method of determining the reference relationship is not limited to this.

[0109] Furthermore, the attribute information decoding process may include at least one of an inverse quantization process, a prediction process, and an arithmetic decoding process. In this case, the reference means using a reference node to calculate a predicted value of the attribute information, or using the state of the reference node (e.g., occupancy information indicating whether the reference node includes a point group) to determine a decoding parameter. For example, the decoding parameter is a quantization parameter in an inverse quantization process, or a context in an arithmetic decoding process.

[0110] The additional information decoding unit 4644 generates additional information by decoding the encoded additional information. The first decoding unit 4640 uses the additional information necessary for decoding the position information and attribute information during decoding, and outputs the additional information necessary for the application to the outside.

[0111] Next, an example of the configuration of the position information encoding unit will be described. Fig. 9 is a block diagram of position information encoding unit 2700 according to this embodiment. Position information encoding unit 2700 includes an octree generation unit 2701, a geometric information calculation unit 2702, a coding table selection unit 2703, and an entropy encoding unit 2704.

[0112] The octree generation unit 2701 generates, for example, an occupancy tree from the input position information, and generates an occupancy code for each node in the occupancy tree. The geometric information calculation unit 2702 acquires information indicating whether an adjacent node of the target node is an occupied node. For example, the geometric information calculation unit 2702 calculates the occupancy information of the adjacent node (information indicating whether the adjacent node is an occupied node) from the occupancy code of the parent node to which the target node belongs. The geometric information calculation unit 2702 may also store encoded nodes in a list and search for adjacent nodes from the list. The geometric information calculation unit 2702 may also switch adjacent nodes depending on the position of the target node within the parent node.

[0113] The coding table selection unit 2703 selects a coding table to be used for entropy coding of the target node using the occupancy information of the adjacent nodes calculated by the geometric information calculation unit 2702. For example, the coding table selection unit 2703 may generate a bit string using the occupancy information of the adjacent nodes, and select a coding table for an index number generated from the bit string.

[0114] The entropy coding unit 2704 generates the coding position information and metadata by entropy coding the occupancy code of the target node using the coding table of the selected index number. The entropy coding unit 2704 may add information indicating the selected coding table to the coding position information.

[0115] The octree representation and the scanning order of position information will be explained below. Position information (position data) is converted (octreeized) into an octree structure and then encoded. The octree structure consists of nodes and leaves. Each node has eight nodes or leaves, and each leaf has voxel (VXL) information. Fig. 10 is a diagram showing an example of the structure of position information containing multiple voxels. Fig. 11 is a diagram showing an example of the position information shown in Fig. 10 converted into an octree structure. Here, among the leaves shown in Fig. 11, leaves 1, 2, and 3 represent voxels VXL1, VXL2, and VXL3 shown in Fig. 10, respectively, and represent a VXL containing a point cloud (hereinafter referred to as effective VXL).

[0116] Specifically, node 1 corresponds to the entire space including the position information in Figure 10. The entire space corresponding to node 1 is divided into eight nodes, and of the eight nodes, the node containing a valid VXL is further divided into eight nodes or leaves, and this process is repeated for each level of the tree structure. Here, each node corresponds to a subspace, and has information (occupancy code) as node information indicating at what position the next node or leaf will be located after division. In addition, the block at the bottom level is set as a leaf, and the number of point clouds contained in the leaf, etc., is held as leaf information.

[0117] Next, an example of the configuration of the position information decoding unit will be described. Fig. 12 is a block diagram of the position information decoding unit 2710 according to this embodiment. The position information decoding unit 2710 includes an octree generation unit 2711, a geometric information calculation unit 2712, a coding table selection unit 2713, and an entropy decoding unit 2714.

[0118] The octree generation unit 2711 generates an octree of a certain space (node) using header information or metadata of a bitstream. For example, the octree generation unit 2711 generates a large space (root node) using the sizes of the x-axis, y-axis, and z-axis directions of a certain space added to the header information, and generates an octree by dividing the space into two in the x-axis, y-axis, and z-axis directions to generate eight small spaces A (nodes A0 to A7). In addition, nodes A0 to A7 are set in order as target nodes.

[0119] The geometric information calculation unit 2712 acquires occupancy information indicating whether an adjacent node of the target node is an occupied node. For example, the geometric information calculation unit 2712 calculates the occupancy information of the adjacent node from the occupancy code of the parent node to which the target node belongs. The geometric information calculation unit 2712 may also store decoded nodes in a list and search for adjacent nodes from the list. The geometric information calculation unit 2712 may also switch adjacent nodes depending on the position of the target node within the parent node.

[0120] The coding table selection unit 2713 selects a coding table (decoding table) to be used for entropy decoding of the target node using the occupancy information of the adjacent nodes calculated by the geometric information calculation unit 2712. For example, the coding table selection unit 2713 may generate a bit string using the occupancy information of the adjacent nodes, and select a coding table for an index number generated from the bit string.

[0121] The entropy decoding unit 2714 generates position information by entropy decoding the occupancy code of the target node using the selected coding table. Note that the entropy decoding unit 2714 may obtain information on the selected coding table by decoding it from the bitstream, and entropy decode the occupancy code of the target node using the coding table indicated by the information.

[0122] The configurations of the attribute information encoding unit and the attribute information decoding unit will be described below. Fig. 13 is a block diagram showing an example configuration of the attribute information encoding unit A100. The attribute information encoding unit may include multiple encoding units that execute different encoding methods. For example, the attribute information encoding unit may switch between the following two methods depending on the use case:

[0123] The attribute information encoding unit A100 includes an LoD attribute information encoding unit A101 and a conversion attribute information encoding unit A102. The LoD attribute information encoding unit A101 classifies each 3D point into multiple layers using position information of the 3D points, predicts attribute information of the 3D points belonging to each layer, and encodes the prediction residual. Here, each classified layer is called an LoD (Level of Detail).

[0124] The transformed attribute information encoding unit A102 encodes the attribute information using RAHT (Region Adaptive Hierarchical Transform). Specifically, the transformed attribute information encoding unit A102 applies RAHT or Haar transform to each piece of attribute information based on the position information of the three-dimensional point to generate high-frequency components and low-frequency components for each layer, and encodes these values ​​using quantization, entropy coding, etc.

[0125] 14 is a block diagram showing an example configuration of the attribute information decoding unit A110. The attribute information decoding unit may include multiple decoding units that execute different decoding methods. For example, the attribute information decoding unit may switch between the following two methods for decoding based on information included in the header or metadata:

[0126] The attribute information decoding unit A110 includes an LoD attribute information decoding unit A111 and a converted attribute information decoding unit A112. The LoD attribute information decoding unit A111 classifies each 3D point into multiple layers using the position information of the 3D points, and decodes the attribute values ​​while predicting the attribute information of the 3D points belonging to each layer.

[0127] The transformed attribute information decoding unit A112 decodes the attribute information using RAHT (Region Adaptive Hierarchical Transform). Specifically, the transformed attribute information decoding unit A112 decodes the attribute values ​​by applying inverse RAHT or inverse Haar transform to the high-frequency components and low-frequency components of each attribute value based on the position information of the three-dimensional point.

[0128] FIG. 15 is a block diagram showing the configuration of an attribute information encoding unit 3140, which is an example of the LoD attribute information encoding unit A101.

[0129] The attribute information encoding unit 3140 includes an LoD generation unit 3141, a surrounding search unit 3142, a prediction unit 3143, a prediction residual calculation unit 3144, a quantization unit 3145, an arithmetic encoding unit 3146, an inverse quantization unit 3147, a decoded value generation unit 3148, and a memory 3149.

[0130] The LoD generation unit 3141 generates LoD using the position information of the three-dimensional points.

[0131] The surrounding search unit 3142 searches for nearby 3D points adjacent to each 3D point using the LoD generation result by the LoD generation unit 3141 and distance information indicating the distance between each 3D point.

[0132] The prediction unit 3143 generates a predicted value of the attribute information of the target 3D point to be coded.

[0133] The prediction residual calculation unit 3144 calculates (generates) a prediction residual of the predicted value of the attribute information generated by the prediction unit 3143.

[0134] The quantization unit 3145 quantizes the prediction residual of the attribute information calculated by the prediction residual calculation unit 3144 .

[0135] The arithmetic coding unit 3146 arithmetically codes the prediction residuals after being quantized by the quantization unit 3145. The arithmetic coding unit 3146 outputs a bitstream including the arithmetically coded prediction residuals to, for example, a three-dimensional data decoding device.

[0136] Note that the prediction residual may be binarized by, for example, the quantization unit 3145 before being arithmetically coded by the arithmetic coding unit 3146.

[0137] Furthermore, for example, the arithmetic coding unit 3146 may initialize a coding table used for arithmetic coding before arithmetic coding. The arithmetic coding unit 3146 may initialize a coding table used for arithmetic coding for each layer. Furthermore, the arithmetic coding unit 3146 may output information indicating the position of the layer for which the coding table has been initialized, by including it in the bitstream.

[0138] The inverse quantization unit 3147 inverse quantizes the prediction residuals quantized by the quantization unit 3145 .

[0139] The decoded value generation unit 3148 generates a decoded value by adding the predicted value of the attribute information generated by the prediction unit 3143 and the prediction residual after inverse quantization by the inverse quantization unit 3147.

[0140] The memory 3149 is a memory that stores the decoded values ​​of the attribute information of each 3D point decoded by the decoded value generation unit 3148. For example, when generating a predicted value of a 3D point that has not yet been encoded, the prediction unit 3143 generates the predicted value by using the decoded values ​​of the attribute information of each 3D point stored in the memory 3149.

[0141] 16 is a block diagram of an attribute information encoding unit 6600, which is an example of the transformed attribute information encoding unit A102. The attribute information encoding unit 6600 includes a sorting unit 6601, a Haar transform unit 6602, a quantization unit 6603, an inverse quantization unit 6604, an inverse Haar transform unit 6605, a memory 6606, and an arithmetic encoding unit 6607.

[0142] The sorting unit 6601 generates a Morton code using the position information of the 3D points and sorts the multiple 3D points in Morton code order. The Haar transform unit 6602 generates coding coefficients by applying a Haar transform to the attribute information. The quantization unit 6603 quantizes the coding coefficients of the attribute information.

[0143] The inverse quantization unit 6604 inversely quantizes the quantized coding coefficients. The inverse Haar transform unit 6605 applies an inverse Haar transform to the coding coefficients. The memory 6606 stores values ​​of attribute information of multiple decoded 3D points. For example, the attribute information of the decoded 3D points stored in the memory 6606 may be used for predicting uncoded 3D points.

[0144] The arithmetic coding unit 6607 calculates ZeroCnt from the quantized coding coefficients and arithmetically codes the ZeroCnt. The arithmetic coding unit 6607 also arithmetically codes the quantized non-zero coding coefficients. The arithmetic coding unit 6607 may binarize the coding coefficients before arithmetic coding. The arithmetic coding unit 6607 may also generate and code various header information.

[0145] FIG. 17 is a block diagram showing the configuration of an attribute information decoding unit 3150, which is an example of the LoD attribute information decoding unit A111.

[0146] The attribute information decoding unit 3150 includes an LoD generation unit 3151 , a surrounding search unit 3152 , a prediction unit 3153 , an arithmetic decoding unit 3154 , an inverse quantization unit 3155 , a decoded value generation unit 3156 , and a memory 3157 .

[0147] The LoD generation unit 3151 generates LoD using the position information of the 3D points decoded by the position information decoding unit (not shown in FIG. 17).

[0148] The surrounding search unit 3152 searches for nearby 3D points adjacent to each 3D point using the LoD generation result by the LoD generation unit 3151 and distance information indicating the distance between each 3D point.

[0149] The prediction unit 3153 generates a predicted value of the attribute information of the target 3D point to be decoded.

[0150] The arithmetic decoding unit 3154 arithmetically decodes prediction residuals in the bitstream acquired from the attribute information coding unit 3140 shown in FIG. 15. The arithmetic decoding unit 3154 may initialize a decoding table used for arithmetic decoding. The arithmetic decoding unit 3154 initializes a decoding table used for arithmetic decoding for a layer on which the arithmetic coding unit 3146 shown in FIG. 15 has performed coding processing. The arithmetic decoding unit 3154 may initialize a decoding table used for arithmetic decoding for each layer. The arithmetic decoding unit 3154 may initialize the decoding table based on information included in the bitstream that indicates the position of the layer for which the coding table has been initialized.

[0151] The inverse quantization unit 3155 inverse quantizes the prediction residuals arithmetically decoded by the arithmetic decoding unit 3154 .

[0152] The decoded value generation unit 3156 generates a decoded value by adding the predicted value generated by the prediction unit 3153 and the prediction residual after inverse quantization by the inverse quantization unit 3155. The decoded value generation unit 3156 outputs the decoded attribute information data to another device.

[0153] The memory 3157 is a memory that stores the decoded values ​​of the attribute information of each 3D point decoded by the decoded value generation unit 3156. For example, when generating a predicted value of a 3D point that has not yet been decoded, the prediction unit 3153 generates the predicted value by using the decoded value of the attribute information of each 3D point stored in the memory 3157.

[0154] 18 is a block diagram of an attribute information decoding unit 6610, which is an example of the transformed attribute information decoding unit A112. The attribute information decoding unit 6610 includes an arithmetic decoding unit 6611, an inverse quantization unit 6612, an inverse Haar transform unit 6613, and a memory 6614.

[0155] The arithmetic decoding unit 6611 arithmetically decodes the ZeroCnt and coding coefficients included in the bitstream. Note that the arithmetic decoding unit 6611 may also decode various types of header information.

[0156] The inverse quantization unit 6612 inverse quantizes the arithmetically decoded coding coefficients. The inverse Haar transform unit 6613 applies inverse Haar transform to the inversely quantized coding coefficients. The memory 6614 stores values ​​of attribute information of multiple decoded 3D points. For example, the attribute information of the decoded 3D points stored in the memory 6614 may be used to predict undecoded 3D points.

[0157] Next, a description will be given of second encoding unit 4650, which is an example of encoding unit 4613 that performs encoding using the second encoding method. Fig. 19 is a diagram showing the configuration of second encoding unit 4650. Fig. 20 is a block diagram of second encoding unit 4650.

[0158] The second encoding unit 4650 generates encoded data (encoded stream) by encoding the point cloud data using a second encoding method. The second encoding unit 4650 includes an additional information generation unit 4651, a position image generation unit 4652, an attribute image generation unit 4653, a video encoding unit 4654, an additional information encoding unit 4655, and a multiplexing unit 4656.

[0159] The second encoding unit 4650 has a feature of generating a position image and an attribute image by projecting a three-dimensional structure onto a two-dimensional image, and encoding the generated position image and attribute image using an existing video encoding method. The second encoding method is also called VPCC (Video based PCC).

[0160] The point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData).

[0161] The additional information generating unit 4651 generates map information of a plurality of two-dimensional images by projecting a three-dimensional structure onto the two-dimensional images.

[0162] The position image generation unit 4652 generates a position image (Geometry Image) based on the position information and the map information generated by the additional information generation unit 4651. This position image is, for example, a distance image in which distance (Depth) is indicated as a pixel value. Note that this distance image may be an image in which multiple point clouds are viewed from a single viewpoint (an image in which multiple point clouds are projected onto a single two-dimensional plane), or may be multiple images in which multiple point clouds are viewed from multiple viewpoints, or may be a single image in which these multiple images are integrated.

[0163] The attribute image generation unit 4653 generates an attribute image based on the attribute information and the map information generated by the additional information generation unit 4651. This attribute image is, for example, an image in which attribute information (for example, color (RGB)) is represented as pixel values. Note that this image may be an image in which multiple point clouds are viewed from one viewpoint (an image in which multiple point clouds are projected onto one two-dimensional plane), or multiple images in which multiple point clouds are viewed from multiple viewpoints, or a single image in which these multiple images are integrated.

[0164] The video encoding unit 4654 generates an encoded position image (Compressed Geometry Image) and an encoded attribute image (Compressed Attribute Image), which are encoded data, by encoding the position image and the attribute image using a video encoding method. Note that any known encoding method may be used as the video encoding method. For example, the video encoding method is AVC, HEVC, or the like.

[0165] The additional information encoding unit 4655 generates encoded additional information (Compressed MetaData) by encoding the additional information, map information, and the like included in the point cloud data.

[0166] The multiplexing unit 4656 multiplexes the encoding position image, the encoding attribute image, the encoding additional information, and other additional information to generate an encoded stream (Compressed Stream) that is encoded data. The generated encoded stream is output to a processing unit in a system layer (not shown).

[0167] Next, a second decoding unit 4660, which is an example of the decoding unit 4624 that performs decoding using the second encoding method, will be described. FIG. 21 is a diagram showing the configuration of the second decoding unit 4660. FIG. 22 is a block diagram of the second decoding unit 4660. The second decoding unit 4660 generates point cloud data by decoding, using the second encoding method, coded data (coded stream) that has been coded using the second coding method. The second decoding unit 4660 includes a demultiplexing unit 4661, a video decoding unit 4662, an additional information decoding unit 4663, a position information generation unit 4664, and an attribute information generation unit 4665.

[0168] A coded stream (compressed stream) that is coded data is input to the second decoding unit 4660 from a processing unit in a system layer (not shown).

[0169] The demultiplexer 4661 separates the encoded position image (Compressed Geometry Image), the encoded attribute image (Compressed Attribute Image), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0170] The video decoding unit 4662 generates a position image and an attribute image by decoding the encoded position image and the encoded attribute image using a video encoding method. Note that any known encoding method may be used as the video encoding method. For example, the video encoding method is AVC or HEVC.

[0171] The additional information decoding unit 4663 decodes the encoded additional information to generate additional information including map information and the like.

[0172] The position information generating unit 4664 generates position information using the position image and map information. The attribute information generating unit 4665 generates attribute information using the attribute image and map information.

[0173] The second decoding unit 4660 uses the additional information necessary for decoding during decoding, and outputs the additional information necessary for the application to the outside.

[0174] The following describes the problems with the PCC encoding method. Fig. 23 is a diagram showing a protocol stack related to PCC encoded data. Fig. 23 shows an example in which other media data such as video (e.g., HEVC) or audio is multiplexed with PCC encoded data and transmitted or stored.

[0175] Multiplexing methods and file formats have the function of multiplexing various coded data and transmitting or storing them. To transmit or store coded data, the coded data must be converted into the format of the multiplexing method. For example, HEVC specifies a technology that stores coded data in a data structure called a NAL unit and stores the NAL unit in ISOBMFF.

[0176] On the other hand, currently, a first encoding method (Codec1) and a second encoding method (Codec2) are being considered as methods for encoding point cloud data, but the structure of the encoded data and the method for storing the encoded data in a system format have not been defined, which poses the problem that, as it stands, it is not possible to perform MUX processing (multiplexing) in the encoding unit, transmission, or storage.

[0177] In the following description, unless a specific encoding method is specified, it refers to either the first encoding method or the second encoding method.

[0178] (Embodiment 2) In this embodiment, the types of coded data (position information (Geometry), attribute information (Attribute), additional information (Metadata)) generated by the first coding unit 4630 or the second coding unit 4650 described above, a method for generating the additional information (Metadata), and multiplexing processing in the multiplexing unit will be described. Note that the additional information (Metadata) may also be referred to as a parameter set or control information.

[0179] In this embodiment, the dynamic object (three-dimensional point cloud data that changes over time) described in Figure 4 will be used as an example, but a similar method may also be used in the case of a static object (three-dimensional point cloud data at any time).

[0180] 24 is a diagram showing the configurations of an encoding unit 4801 and a multiplexing unit 4802 included in the three-dimensional data encoding device according to this embodiment. The encoding unit 4801 corresponds to, for example, the first encoding unit 4630 or the second encoding unit 4650 described above. The multiplexing unit 4802 corresponds to the multiplexing unit 4634 or 4656 described above.

[0181] The encoding unit 4801 encodes point cloud data of a plurality of PCC (Point Cloud Compression) frames, and generates encoded data (Multiple Compressed Data) of a plurality of pieces of position information, attribute information, and additional information.

[0182] The multiplexing unit 4802 converts data of multiple data types (position information, attribute information, and additional information) into NAL units, thereby converting the data into a data structure that takes into account data access in the decoding device.

[0183] 25 is a diagram showing an example of the structure of coded data generated by coding unit 4801. Arrows in the diagram indicate dependencies related to the decoding of coded data, with the source of the arrow depending on the data at the end of the arrow. In other words, the decoding device decodes the data at the end of the arrow and uses the decoded data to decode the data at the end of the arrow. In other words, dependency means that the data on which the dependency is based is referenced (used) in the processing (encoding, decoding, etc.) of the data on which the dependency is based.

[0184] First, the process of generating encoded data of position information will be described. The encoding unit 4801 generates encoded position data (compressed geometry data) for each frame by encoding the position information of each frame. The encoded position data is represented by G(i), where i indicates the frame number, the time of the frame, etc.

[0185] The encoding unit 4801 also generates a position parameter set (GPS(i)) corresponding to each frame. The position parameter set includes parameters that can be used to decode the encoded position data. The encoded position data for each frame depends on the corresponding position parameter set.

[0186] Furthermore, encoded position data consisting of multiple frames is defined as a position sequence (Geometry Sequence). The encoding unit 4801 generates a position sequence parameter set (Geometry Sequence PS: also referred to as position SPS) that stores parameters commonly used in decoding processes for multiple frames in the position sequence. The position sequence depends on the position SPS.

[0187] Next, the process of generating coded data of attribute information will be described. The coding unit 4801 generates coded attribute data (Compressed Attribute Data) for each frame by coding the attribute information of each frame. The coded attribute data is represented by A(i). FIG. 25 shows an example in which attribute X and attribute Y exist, and the coded attribute data of attribute X is represented by AX(i) and the coded attribute data of attribute Y is represented by AY(i).

[0188] The encoding unit 4801 also generates an attribute parameter set (APS(i)) corresponding to each frame. The attribute parameter set for attribute X is represented as AXPS(i), and the attribute parameter set for attribute Y is represented as AYPS(i). The attribute parameter set includes parameters that can be used to decode encoded attribute information. The encoded attribute data depends on the corresponding attribute parameter set.

[0189] Furthermore, encoded attribute data consisting of multiple frames is defined as an attribute sequence. The encoding unit 4801 generates an attribute sequence parameter set (Attribute Sequence PS: also referred to as attribute SPS) that stores parameters commonly used in decoding processes for multiple frames in the attribute sequence. The attribute sequence depends on the attribute SPS.

[0190] Furthermore, in the first encoding method, the encoded attribute data depends on the encoded position data.

[0191] 25 shows an example in which two types of attribute information (attribute X and attribute Y) exist. When there are two types of attribute information, for example, two encoding units generate respective data and metadata. Also, for example, an attribute sequence is defined for each type of attribute information, and an attribute SPS is generated for each type of attribute information.

[0192] 25 shows an example in which there is one type of position information and two types of attribute information, but this is not limiting, and there may be one type of attribute information, or three or more types. In this case, encoded data can be generated using a similar method. Furthermore, in the case of point cloud data that does not have attribute information, the attribute information may not be necessary. In this case, the encoding unit 4801 does not need to generate a parameter set related to the attribute information.

[0193] Next, the process of generating additional information (metadata) will be described. The encoding unit 4801 generates a PCC Stream PS (also referred to as stream PS), which is a parameter set for the entire PCC stream. The encoding unit 4801 stores, in the stream PS, parameters that can be used in common for decoding processes for one or more position sequences and one or more attribute sequences. For example, the stream PS includes identification information indicating the codec for the point cloud data, information indicating the algorithm used for encoding, and the like. The position sequence and attribute sequence depend on the stream PS.

[0194] Next, the access unit and GOF will be explained. In this embodiment, the concepts of the access unit (AU) and GOF (Group of Frame) are newly introduced.

[0195] An access unit is a basic unit for accessing data during decoding, and is composed of one or more pieces of data and one or more pieces of metadata. For example, an access unit is composed of position information at the same time and one or more pieces of attribute information. A GOF is a random access unit and is composed of one or more access units.

[0196] The encoding unit 4801 generates an access unit header (AU Header) as identification information indicating the beginning of an access unit. The encoding unit 4801 stores parameters related to the access unit in the access unit header. For example, the access unit header includes the configuration or information of the coded data included in the access unit. The access unit header also includes parameters commonly used for the data included in the access unit, such as parameters related to decoding of the coded data.

[0197] Instead of an access unit header, the encoding unit 4801 may generate an access unit delimiter that does not include parameters related to the access unit. This access unit delimiter is used as identification information indicating the start of the access unit. The decoding device identifies the start of the access unit by detecting the access unit header or the access unit delimiter.

[0198] Next, the generation of identification information for the start of a GOF will be described. The encoding unit 4801 generates a GOF header as identification information that indicates the start of a GOF. The encoding unit 4801 stores parameters related to the GOF in the GOF header. For example, the GOF header includes the configuration or information of the coded data included in the GOF. The GOF header also includes parameters commonly used for the data included in the GOF, such as parameters related to the decoding of the coded data.

[0199] Instead of a GOF header, the encoding unit 4801 may generate a GOF delimiter that does not include parameters related to the GOF. This GOF delimiter is used as identification information that indicates the beginning of the GOF. The decoding device identifies the beginning of the GOF by detecting the GOF header or the GOF delimiter.

[0200] In PCC encoded data, for example, an access unit is defined as a PCC frame unit, and a decoding device accesses a PCC frame based on identification information at the beginning of the access unit.

[0201] Also, for example, GOF is defined as one random access unit. A decoding device accesses the random access unit based on the identification information at the beginning of the GOF. For example, if PCC frames are not dependent on each other and can be decoded independently, the PCC frames may be defined as the random access unit.

[0202] It should be noted that two or more PCC frames may be allocated to one access unit, and multiple random access units may be allocated to one GOF.

[0203] The encoding unit 4801 may also define and generate parameter sets or metadata other than those described above. For example, the encoding unit 4801 may generate SEI (Supplemental Enhancement Information) that stores parameters (optional parameters) that may not necessarily be used during decoding.

[0204] Next, the structure of coded data and the method of storing coded data in NAL units will be described.

[0205] For example, a data format is defined for each type of coded data. Figure 26 shows examples of coded data and NAL units.

[0206] For example, as shown in Fig. 26, the coded data includes a header and a payload. The coded data may include length information indicating the length (amount of data) of the coded data, the header, or the payload. The coded data may not include a header.

[0207] The header includes, for example, identification information for identifying the data, such as the data type or frame number.

[0208] The header includes, for example, identification information indicating a reference relationship. This identification information is stored in the header when, for example, there is a dependency relationship between data, and is information for referencing the reference destination from the reference source. For example, the header of the reference destination includes identification information for identifying the data. The header of the reference source includes identification information indicating the reference destination.

[0209] If the reference destination or the reference source can be identified or derived from other information, the identification information for specifying the data or the identification information indicating the reference relationship may be omitted.

[0210] The multiplexing unit 4802 stores the coded data in the payload of the NAL unit. The NAL unit header includes pcc_nal_unit_type, which is information identifying the coded data. Figure 27 shows an example of the semantics of pcc_nal_unit_type.

[0211] As shown in Fig. 27, when pcc_codec_type is codec 1 (Codec1: first encoding method), values ​​0 to 10 of pcc_nal_unit_type are assigned to the encoded position data (Geometry), encoded attribute X data (AttributeX), encoded attribute Y data (AttributeY), position PS (Geom.PS), attribute XPS (AttrX.PS), attribute YPS (AttrX.PS), position SPS (Geometry Sequence PS), attribute XSPS (AttributeX Sequence PS), attribute YSPS (AttributeY Sequence PS), AU header (AU Header), and GOF header (GOF Header) in codec 1. Values ​​11 and above are assigned as spares for codec 1.

[0212] When pcc_codec_type is Codec2 (Codec2: second encoding method), values ​​0 to 2 of pcc_nal_unit_type are assigned to codec data A (DataA), metadata A (MetaDataA), and metadata B (MetaDataB). Values ​​3 and above are assigned as spares for Codec2.

[0213] Next, the data transmission order will be explained. The following explains the restrictions on the transmission order of NAL units.

[0214] The multiplexing unit 4802 collectively transmits NAL units in units of GOF or AU. The multiplexing unit 4802 places a GOF header at the beginning of a GOF, and places an AU header at the beginning of an AU.

[0215] The multiplexing unit 4802 may allocate a sequence parameter set (SPS) for each AU so that the decoding device can decode from the next AU even if data is lost due to packet loss or the like.

[0216] If the coded data has a dependency relationship related to decoding, the decoding device decodes the referenced data first, and then decodes the referenced data. In order to enable the decoding device to decode the data in the order in which it was received without rearranging the data, the multiplexing unit 4802 sends the referenced data first.

[0217] 28 is a diagram showing examples of the transmission order of NAL units, showing three examples: position information priority, parameter priority, and data integration.

[0218] The location information priority transmission order is an example in which information related to location information and information related to attribute information are transmitted together. In this transmission order, the transmission of information related to location information is completed earlier than the transmission of information related to attribute information.

[0219] For example, by using this transmission order, a decoding device that does not decode attribute information may be able to set a time during which it does not process the attribute information by ignoring the decoding of the attribute information. Also, for example, in the case of a decoding device that wants to decode position information quickly, it may be able to decode the position information more quickly by obtaining the encoded data of the position information early.

[0220] In FIG. 28, the attributes XSPS and YSPS are combined and written as the attribute SPS, but the attributes XSPS and YSPS may be arranged separately.

[0221] In the parameter set priority sending order, the parameter sets are sent first and the data is sent later.

[0222] As long as the NAL unit transmission order constraints are met as described above, the multiplexing unit 4802 may transmit NAL units in any order. For example, order identification information may be defined, and the multiplexing unit 4802 may have the function of transmitting NAL units in multiple order patterns. For example, the NAL unit order identification information may be stored in the stream PS.

[0223] The three-dimensional data decoding device may perform decoding based on the order identification information. The three-dimensional data decoding device may instruct the three-dimensional data encoding device on a desired transmission order, and the three-dimensional data encoding device (multiplexing unit 4802) may control the transmission order in accordance with the instructed transmission order.

[0224] The multiplexing unit 4802 may generate coded data that merges multiple functions, as long as the data transmission order is within the constraints of the transmission order, such as the transmission order of the integrated data. For example, as shown in Fig. 28, the GOF header and the AU header may be integrated, or the AXPS and the AYPS may be integrated. In this case, an identifier indicating that the data has multiple functions is defined in pcc_nal_unit_type.

[0225] A modified example of this embodiment will be described below. PS has levels, such as a frame-level PS, a sequence-level PS, and a PCC sequence-level PS, and if the PCC sequence level is the higher level and the frame level is the lower level, the following method may be used to store parameters.

[0226] The default PS value is indicated in the higher PS. Also, if the value of the lower PS differs from the value of the higher PS, the PS value is indicated in the lower PS. Alternatively, the PS value is not written in the higher PS, but written in the lower PS. Alternatively, information on whether the PS value is to be written in the lower PS, the higher PS, or both is written in either the lower PS or the higher PS, or both. Alternatively, the lower PS may be merged with the higher PS. Alternatively, if the lower PS and the higher PS overlap, the multiplexing unit 4802 may omit sending one of them.

[0227] The encoding unit 4801 or the multiplexing unit 4802 may divide the data into slices or tiles and transmit the divided data. The divided data includes information for identifying the divided data, and parameters used to decode the divided data are included in the parameter set. In this case, an identifier indicating that the data stores data or parameters related to tiles or slices is defined in pcc_nal_unit_type.

[0228] The processing related to the order identification information will be explained below. Fig. 29 is a flowchart of processing by the three-dimensional data encoding device (the encoding unit 4801 and the multiplexing unit 4802) related to the transmission order of NAL units.

[0229] First, the three-dimensional data encoding device determines the transmission order of NAL units (position information priority or parameter set priority) (S4801). For example, the three-dimensional data encoding device determines the transmission order based on a specification from a user or an external device (e.g., a three-dimensional data decoding device).

[0230] If the determined transmission order is positional information priority (positional information priority in S4802), the three-dimensional data encoding device sets the order identification information included in the stream PS to positional information priority (S4803). That is, in this case, the order identification information indicates that the NAL units are to be transmitted in an order that prioritizes positional information. The three-dimensional data encoding device then transmits the NAL units in positional information priority order (S4804).

[0231] On the other hand, if the determined transmission order is parameter set priority (parameter set priority in S4802), the three-dimensional data encoding device sets the order identification information included in the stream PS to parameter set priority (S4805). That is, in this case, the order identification information indicates that the NAL units will be transmitted in parameter set priority order. Then, the three-dimensional data encoding device transmits the NAL units in parameter set priority order (S4806).

[0232] 30 is a flowchart of processing by the 3D data decoding device related to the transmission order of NAL units. First, the 3D data decoding device analyzes the order identification information included in the stream PS (S4811).

[0233] If the transmission order indicated by the order identification information is position information priority (position information priority in S4812), the three-dimensional data decoding device decodes the NAL units assuming that the transmission order of the NAL units is position information priority (S4813).

[0234] On the other hand, if the transmission order indicated by the order identification information is parameter set priority (parameter set priority in S4812), the three-dimensional data decoding device decodes the NAL units assuming that the transmission order of the NAL units is parameter set priority (S4814).

[0235] For example, if the three-dimensional data decoding device does not decode attribute information, in step S4813, it may acquire NAL units related to position information without acquiring all NAL units, and decode the position information from the acquired NAL units.

[0236] Next, the processing related to the generation of AUs and GOFs will be described. Fig. 31 is a flowchart of processing by the 3D data encoding device (the multiplexing unit 4802) related to the generation of AUs and GOFs in multiplexing NAL units.

[0237] First, the three-dimensional data encoding device determines the type of encoded data (S4821). Specifically, the three-dimensional data encoding device determines whether the encoded data to be processed is the first data of an AU, the first data of a GOF, or other data.

[0238] If the coded data is data at the start of a GOF (GOF start in S4822), the three-dimensional data coding device places a GOF header and an AU header at the start of the coded data belonging to the GOF to generate a NAL unit (S4823).

[0239] If the coded data is data at the beginning of an AU (AU beginning in S4822), the three-dimensional data coding device places an AU header at the beginning of the coded data belonging to the AU and generates a NAL unit (S4824).

[0240] If the encoded data is not at the beginning of a GOF or an AU (other than the beginning of a GOF or an AU in S4822), the three-dimensional data encoding device places the encoded data after the AU header of the AU to which the encoded data belongs and generates a NAL unit (S4825).

[0241] Next, a process for accessing AUs and GOFs will be described. Fig. 32 is a flowchart of a process performed by a three-dimensional data decoding device for accessing AUs and GOFs during demultiplexing of NAL units.

[0242] First, the three-dimensional data decoding device determines the type of coded data included in the NAL unit by analyzing the nal_unit_type included in the NAL unit (S4831). Specifically, the three-dimensional data decoding device determines whether the coded data included in the NAL unit is data at the beginning of an AU, data at the beginning of a GOF, or other data.

[0243] If the encoded data contained in the NAL unit is the data at the start of the GOF (start of GOF in S4832), the three-dimensional data decoding device determines that the NAL unit is the start position of random access, accesses the NAL unit, and starts the decoding process (S4833).

[0244] On the other hand, if the coded data contained in the NAL unit is data at the beginning of an AU (AU beginning in S4832), the three-dimensional data decoding device determines that the NAL unit is the beginning of an AU, accesses the data contained in the NAL unit, and decodes the AU (S4834).

[0245] On the other hand, if the coded data included in the NAL unit is neither the start of a GOF nor the start of an AU (other than the start of a GOF or an AU in S4832), the three-dimensional data decoding device does not process the NAL unit.

[0246] (Embodiment 3) In this embodiment, a method for representing three-dimensional points (point cloud) in encoding three-dimensional data will be described.

[0247] 33 is a block diagram showing the configuration of a three-dimensional data distribution system according to this embodiment. The distribution system shown in FIG. 33 includes a server 1501 and a plurality of clients 1502.

[0248] Server 1501 includes a storage unit 1511 and a control unit 1512. Storage unit 1511 stores an encoded three-dimensional map 1513, which is encoded three-dimensional data.

[0249] FIG. 34 is a diagram showing an example of the bitstream configuration of the encoded 3D map 1513. The 3D map is divided into multiple sub-maps, and each sub-map is encoded. A random access header (RA) containing sub-coordinate information is attached to each sub-map. The sub-coordinate information is used to improve the encoding efficiency of the sub-map. This sub-coordinate information indicates the sub-coordinate of the sub-map. The sub-coordinate is the coordinate of the sub-map based on a reference coordinate. A 3D map including multiple sub-maps is called an overall map. The reference coordinate (e.g., the origin) in the overall map is called the reference coordinate. In other words, the sub-coordinate is the coordinate of the sub-map in the coordinate system of the overall map. In other words, the sub-coordinate indicates the offset between the coordinate system of the overall map and the coordinate system of the sub-map. The coordinate in the coordinate system of the overall map based on the reference coordinate is called the overall coordinate. The coordinate in the coordinate system of the sub-map based on the sub-coordinate is called the differential coordinate.

[0250] Client 1502 sends a message to server 1501. This message includes location information of client 1502. Control unit 1512 included in server 1501 obtains a bitstream of a submap located closest to the location of client 1502 based on the location information included in the received message. The bitstream of the submap includes subcoordinate information and is sent to client 1502. Decoder 1521 included in client 1502 uses this subcoordinate information to obtain the overall coordinates of the submap relative to the reference coordinates. Application 1522 included in client 1502 executes an application related to its own location using the obtained overall coordinates of the submap.

[0251] Furthermore, a submap indicates a partial area of ​​the overall map. Subcoordinates are the coordinates at which the submap is located in the reference coordinate space of the overall map. For example, suppose that an overall map A has submap A of AA and submap B of AB. When a vehicle wants to refer to the map of AA, it starts decoding from submap A, and when it wants to refer to the map of AB, it starts decoding from submap B. Here, submaps are random access points. Specifically, A is Osaka Prefecture, AA is Osaka City, and AB is Takatsuki City, etc.

[0252] Each submap is transmitted to the client together with sub-coordinate information, which is included in the header information of each submap, a transmission packet, or the like.

[0253] The reference coordinates that serve as the reference coordinates for the sub-coordinate information of each sub-map may be added to header information of a space higher than the sub-map, such as the header information of the overall map.

[0254] A submap may consist of one space (SPC), or it may consist of multiple SPCs.

[0255] A submap may also include a GOS (Group of Space). A submap may also consist of a world. For example, if a submap contains multiple objects, assigning the objects to different SPCs will result in the submap consisting of multiple SPCs. Assigning the objects to a single SPC will result in the submap consisting of a single SPC.

[0256] Next, the effect of improving encoding efficiency when using sub-coordinate information will be described. FIG. 35 is a diagram for explaining this effect. For example, a large number of bits is required to encode 3D point A, which is located far from the reference coordinates shown in FIG. 35. Here, the distance between the sub-coordinates and 3D point A is shorter than the distance between the reference coordinates and 3D point A. Therefore, encoding efficiency can be improved by encoding the coordinates of 3D point A based on the sub-coordinates, rather than encoding the coordinates of 3D point A based on the reference coordinates. Furthermore, the sub-map bitstream includes sub-coordinate information. By sending the sub-map bitstream and the reference coordinates to the decoding side (client), the overall coordinates of the sub-map can be restored on the decoding side.

[0257] FIG. 36 is a flowchart of the processing by the server 1501 that transmits the submap.

[0258] First, the server 1501 receives a message including the location information of the client 1502 from the client 1502 (S1501). The control unit 1512 acquires an encoded bitstream of a submap based on the client's location information from the storage unit 1511 (S1502). Then, the server 1501 transmits the encoded bitstream of the submap and the reference coordinates to the client 1502 (S1503).

[0259] FIG. 37 is a flowchart of the processing by the client 1502 that receives the submap.

[0260] First, the client 1502 receives the coded bitstream of the submap and the reference coordinates transmitted from the server 1501 (S1511). Next, the client 1502 obtains the submap and subcoordinate information by decoding the coded bitstream (S1512). Next, the client 1502 restores the differential coordinates in the submap to global coordinates using the reference coordinates and the subcoordinates (S1513).

[0261] Next, an example of the syntax of information related to submaps will be described. In encoding a submap, a three-dimensional data encoding device calculates differential coordinates by subtracting sub-coordinates from the coordinates of each point cloud (three-dimensional point). The three-dimensional data encoding device then encodes the differential coordinates into a bitstream as the value of each point cloud. The encoding device also encodes sub-coordinate information indicating the sub-coordinates as header information of the bitstream. This allows a three-dimensional data decoding device to obtain the overall coordinates of each point cloud. For example, the three-dimensional data encoding device may be included in server 1501, and the three-dimensional data decoding device may be included in client 1502.

[0262] Figure 38 is a diagram showing an example of the syntax of a submap. NumOfPoint shown in Figure 38 indicates the number of point clouds included in the submap. sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z are sub-coordinate information. sub_coordinate_x indicates the x-coordinate of the sub-coordinate. sub_coordinate_y indicates the y-coordinate of the sub-coordinate. sub_coordinate_z indicates the z-coordinate of the sub-coordinate.

[0263] Also, diff_x[i], diff_y[i], and diff_z[i] are the differential coordinates of the i-th point cloud in the submap. diff_x[i] indicates the differential value between the x-coordinate of the i-th point cloud in the submap and the x-coordinate of the sub-coordinate. diff_y[i] indicates the differential value between the y-coordinate of the i-th point cloud in the submap and the y-coordinate of the sub-coordinate. diff_z[i] indicates the differential value between the z-coordinate of the i-th point cloud in the submap and the z-coordinate of the sub-coordinate.

[0264] The three-dimensional data decoding device decodes point_cloud[i]_x, point_cloud[i]_y, and point_cloud[i]_z, which are the global coordinates of the i-th point cloud, using the following equations: point_cloud[i]_x is the x coordinate of the global coordinates of the i-th point cloud; point_cloud[i]_y is the y coordinate of the global coordinates of the i-th point cloud; and point_cloud[i]_z is the z coordinate of the global coordinates of the i-th point cloud.

[0265] point_cloud[i]_x=sub_coordinate_x+diff_x[i] point_cloud[i]_y=sub_coordinate_y+diff_y[i] point_cloud[i]_z=sub_coordinate_z+diff_z[i]

[0266] Next, the process of switching the application of octree coding will be described. When encoding a submap, the three-dimensional data encoding device selects whether to encode each point cloud using an octree representation (hereinafter referred to as octree coding) or to encode differential values ​​from sub-coordinates (hereinafter referred to as non-octree coding). FIG. 39 is a diagram schematically illustrating this operation. For example, if the number of point clouds in a submap is equal to or greater than a predetermined threshold, the three-dimensional data encoding device applies octree coding to the submap. If the number of point clouds in a submap is less than the threshold, the three-dimensional data encoding device applies non-octree coding to the submap. This allows the three-dimensional data encoding device to appropriately select whether to use octree coding or non-octree coding depending on the shape and density of objects included in the submap, thereby improving encoding efficiency.

[0267] Furthermore, the three-dimensional data encoding device adds information indicating whether octree encoding or non-octree encoding has been applied to the submap (hereinafter referred to as octree encoding application information) to the header of the submap, etc. This allows the three-dimensional data decoding device to determine whether the bitstream is a bitstream obtained by octree encoding the submap or a bitstream obtained by non-octree encoding the submap.

[0268] In addition, the three-dimensional data encoding device may calculate the encoding efficiency when applying octree encoding and non-octree encoding to the same point cloud, and apply the encoding method with the highest encoding efficiency to the submap.

[0269] Figure 40 is a diagram showing an example of the syntax of a submap when this switching is performed. coding_type shown in Figure 40 is information indicating the coding type, and is the above-mentioned octree coding application information. coding_type=00 indicates that octree coding has been applied. coding_type=01 indicates that non-octree coding has been applied. coding_type=10 or 11 indicates that a coding method other than those mentioned above has been applied.

[0270] If the coding type is non-octree coding (non_octree), the submap includes NumOfPoint and sub-coordinate information (sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z).

[0271] If the coding type is octree coding, the submap includes octree_info, which is information necessary for octree coding, such as depth information.

[0272] If the coding type is non_octree, the submap contains difference coordinates (diff_x[i], diff_y[i], and diff_z[i]).

[0273] If the coding type is octree coding, the submap contains octree_data, which is coding data for octree coding.

[0274] Although an example in which the xyz coordinate system is used as the coordinate system of the point cloud has been shown here, a polar coordinate system may also be used.

[0275] 41 is a flowchart of a three-dimensional data encoding process performed by a three-dimensional data encoding device. First, the three-dimensional data encoding device calculates the number of point clouds in a target submap, which is the submap to be processed (S1521). Next, the three-dimensional data encoding device determines whether the calculated number of point clouds is equal to or greater than a predetermined threshold (S1522).

[0276] If the number of point clouds is equal to or greater than the threshold (Yes in S1522), the three-dimensional data encoding device applies octree encoding to the target submap (S1523). In addition, the three-dimensional point data encoding device adds octree encoding application information indicating that octree encoding has been applied to the target submap to the header of the bitstream (S1525).

[0277] On the other hand, if the number of point clouds is less than the threshold (No in S1522), the 3D data encoding device applies non-octree encoding to the target submap (S1524).The 3D point data encoding device also adds octree encoding application information indicating that non-octree encoding has been applied to the target submap to the bitstream header (S1525).

[0278] 42 is a flowchart of three-dimensional data decoding processing by a three-dimensional data decoding device. First, the three-dimensional data decoding device decodes octree coding application information from the header of the bitstream (S1531). Next, the three-dimensional data decoding device determines whether the coding type applied to the target submap is octree coding based on the decoded octree coding application information (S1532).

[0279] If the coding type indicated by the octree coding application information is octree coding (Yes in S1532), the three-dimensional data decoding device decodes the target submap using octree decoding (S1533). On the other hand, if the coding type indicated by the octree coding application information is non-octree coding (No in S1532), the three-dimensional data decoding device decodes the target submap using non-octree decoding (S1534).

[0280] A modification of this embodiment will be described below. Figures 43 to 45 are diagrams schematically showing the operation of a modification of the coding type switching process.

[0281] As shown in Figure 43, the three-dimensional data encoding device may select whether to apply octree encoding or non-octree encoding for each space. In this case, the three-dimensional data encoding device adds octree encoding application information to the header of the space. This allows the three-dimensional data decoding device to determine for each space whether octree encoding has been applied. In this case, the three-dimensional data encoding device also sets sub-coordinates for each space and encodes the difference values ​​obtained by subtracting the values ​​of the sub-coordinates from the coordinates of each point cloud in the space.

[0282] This allows the three-dimensional data encoding device to appropriately switch whether or not to apply octree encoding depending on the shape of the object in the space or the number of point clouds, thereby improving encoding efficiency.

[0283] Furthermore, as shown in Figure 44, the three-dimensional data encoding device may select whether to apply octree encoding or non-octree encoding for each volume. In this case, the three-dimensional data encoding device adds octree encoding application information to the header of the volume. This allows the three-dimensional data decoding device to determine for each volume whether octree encoding has been applied. In this case, the three-dimensional data encoding device also sets sub-coordinates for each volume and encodes the difference values ​​obtained by subtracting the values ​​of the sub-coordinates from the coordinates of each point cloud within the volume.

[0284] This allows the three-dimensional data encoding device to appropriately switch whether or not to apply octree encoding depending on the shape of the object in the volume or the number of point clouds, thereby improving encoding efficiency.

[0285] In the above explanation, an example was given in which the difference obtained by subtracting the sub-coordinates from the coordinates of each point cloud was coded as non-octree coding, but this is not necessarily limited to this, and any coding method other than octree coding may be used. For example, as shown in Fig. 45, the three-dimensional data coding device may use a method (hereinafter referred to as original coordinate coding) in which the values ​​of the point cloud within a submap, space, or volume themselves are coded as non-octree coding, rather than the difference from the sub-coordinates.

[0286] In this case, the three-dimensional data encoding device stores information in the header indicating that original coordinate encoding has been applied to the target space (submap, space, or volume), which allows the three-dimensional data decoding device to determine whether original coordinate encoding has been applied to the target space.

[0287] Furthermore, when applying original coordinate coding, the three-dimensional data coding device may perform coding without applying quantization and arithmetic coding to the original coordinates. Furthermore, the three-dimensional data coding device may code the original coordinates with a predetermined fixed bit length. This allows the three-dimensional data coding device to generate a stream with a constant bit length at a certain timing.

[0288] In the above description, an example has been given in which the difference obtained by subtracting the sub-coordinates from the coordinates of each point cloud is coded as non-octree coding, but the present invention is not necessarily limited to this.

[0289] For example, the three-dimensional data encoding device may sequentially encode the difference values ​​between the coordinates of each point cloud. FIG. 46 is a diagram for explaining the operation in this case. For example, in the example shown in FIG. 46, when encoding point cloud PA, the three-dimensional data encoding device uses sub-coordinates as predicted coordinates and encodes the difference values ​​between the coordinates of point cloud PA and the predicted coordinates. Furthermore, when encoding point cloud PB, the three-dimensional data encoding device uses the coordinates of point cloud PA as predicted coordinates and encodes the difference values ​​between point cloud PB and the predicted coordinates. Furthermore, when encoding point cloud PC, the three-dimensional data encoding device uses point cloud PB as predicted coordinates and encodes the difference values ​​between point cloud PB and the predicted coordinates. In this way, the three-dimensional data encoding device may set a scan order for multiple point clouds and encode the difference values ​​between the coordinates of a target point cloud to be processed and the coordinates of the point cloud immediately preceding the target point cloud in the scan order.

[0290] Furthermore, in the above description, the sub-coordinates are coordinates of the lower left front corner of the sub-map, but the positions of the sub-coordinates are not limited to this. FIGS. 47 to 49 are diagrams showing other examples of the positions of the sub-coordinates. The sub-coordinates may be set to any coordinates within the target space (sub-map, space, or volume). That is, as described above, the sub-coordinates may be coordinates of the lower left front corner of the target space. As shown in FIG. 47, the sub-coordinates may be coordinates of the center of the target space. As shown in FIG. 48, the sub-coordinates may be coordinates of the upper right back corner of the target space. Furthermore, the sub-coordinates are not limited to coordinates of the lower left front or upper right back corner of the target space, but may be coordinates of any corner of the target space.

[0291] In addition, the setting position of the sub-coordinates may be the same as the coordinates of a certain point cloud in the target space (submap, space, or volume). For example, in the example shown in Figure 49, the coordinates of the sub-coordinates match the coordinates of the point cloud PD.

[0292] Furthermore, in this embodiment, an example has been shown in which the application of octree coding and the application of non-octree coding are switched, but this is not necessarily limited to this. For example, the three-dimensional data coding device may switch between applying a tree structure other than an octree and applying a non-tree structure other than the tree structure. For example, the other tree structure may be a kd tree in which division is performed using a plane perpendicular to one of the coordinate axes. Note that any method may be used as the other tree structure.

[0293] Furthermore, although the present embodiment has shown an example in which coordinate information of a point cloud is encoded, this is not necessarily limited to this. The three-dimensional data encoding device may also encode, for example, color information, three-dimensional feature quantities, or visible light feature quantities in the same manner as coordinate information. For example, the three-dimensional data encoding device may set the average value of the color information of each point cloud in a submap as sub-color information (sub-color), and encode the difference between the color information of each point cloud and the sub-color information.

[0294] Furthermore, in this embodiment, an example has been shown in which a coding method (octree coding or non-octree coding) with good coding efficiency is selected depending on the number of point clouds, etc., but this is not necessarily limited to this. For example, a three-dimensional data coding device on the server side may store bit streams of point clouds coded using octree coding, bit streams of point clouds coded using non-octree coding, and bit streams of point clouds coded using both of these coding methods, and switch the bit stream to be sent to the three-dimensional data decoding device depending on the communication environment or the processing capacity of the three-dimensional data decoding device.

[0295] Fig. 50 is a diagram showing an example of volume syntax when switching the application of octree coding. The syntax shown in Fig. 50 is basically the same as the syntax shown in Fig. 40, except that each piece of information is information on a volume basis. Specifically, NumOfPoint indicates the number of point clouds included in the volume. sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z are sub-coordinate information of the volume.

[0296] Also, diff_x[i], diff_y[i], and diff_z[i] are the differential coordinates of the i-th point cloud in the volume. diff_x[i] indicates the differential value between the x-coordinate of the i-th point cloud in the volume and the x-coordinate of the sub-coordinate. diff_y[i] indicates the differential value between the y-coordinate of the i-th point cloud in the volume and the y-coordinate of the sub-coordinate. diff_z[i] indicates the differential value between the z-coordinate of the i-th point cloud in the volume and the z-coordinate of the sub-coordinate.

[0297] If the relative positions of volumes in space can be calculated, the three-dimensional data encoding device does not need to include sub-coordinate information in the volume header. In other words, the three-dimensional data encoding device may calculate the relative positions of volumes in space without including the sub-coordinate information in the header, and use the calculated positions as the sub-coordinates of each volume.

[0298] As described above, the three-dimensional data encoding device according to this embodiment determines whether or not to encode a target spatial unit among multiple spatial units (e.g., submaps, spaces, or volumes) included in three-dimensional data using an octree structure (e.g., S1522 in FIG. 41). For example, if the number of three-dimensional points included in the target spatial unit is greater than a predetermined threshold, the three-dimensional data encoding device determines to encode the target spatial unit using an octree structure. Furthermore, if the number of three-dimensional points included in the target spatial unit is equal to or less than the threshold, the three-dimensional data encoding device determines not to encode the target spatial unit using an octree structure.

[0299] If it is determined that the target space unit is to be coded using an octree structure (Yes in S1522), the three-dimensional data coding device codes the target space unit using the octree structure (S1523). On the other hand, if it is determined that the target space unit is not to be coded using an octree structure (No in S1522), the three-dimensional data coding device codes the target space unit using a method other than the octree structure (S1524). For example, in the different method, the three-dimensional data coding device codes the coordinates of three-dimensional points included in the target space unit. Specifically, in the different method, the three-dimensional data coding device codes the difference between the reference coordinates of the target space unit and the coordinates of three-dimensional points included in the target space unit.

[0300] Next, the three-dimensional data encoding device adds information indicating whether the target spatial unit has been encoded using an octree structure to the bitstream (S1525).

[0301] This allows the three-dimensional data encoding device to reduce the amount of data in the encoded signal, thereby improving encoding efficiency.

[0302] For example, the three-dimensional data encoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0303] Furthermore, the three-dimensional data decoding device according to this embodiment decodes, from the bitstream, information indicating whether or not a target spatial unit among multiple target spatial units (e.g., submaps, spaces, or volumes) included in the three-dimensional data is to be decoded using an octree structure (e.g., S1531 in FIG. 42). If the information indicates that the target spatial unit is to be decoded using an octree structure (Yes in S1532), the three-dimensional data decoding device decodes the target spatial unit using the octree structure (S1533).

[0304] If the information indicates that the target space unit is not to be decoded using an octree structure (No in S1532), the three-dimensional data decoding device decodes the target space unit using a method other than the octree structure (S1534). For example, in the different method, the three-dimensional data decoding device decodes the coordinates of three-dimensional points included in the target space unit. Specifically, in the different method, the three-dimensional data decoding device decodes the difference between the reference coordinates of the target space unit and the coordinates of three-dimensional points included in the target space unit.

[0305] This allows the three-dimensional data decoding device to reduce the amount of data in the coded signal, thereby improving coding efficiency.

[0306] For example, the three-dimensional data decoding device includes a processor and a memory, and the processor performs the above processing using the memory.

[0307] (Fourth embodiment) The information of a 3D point cloud includes position information (geometry) and attribute information (attribute). The position information includes coordinates (x-coordinate, y-coordinate, z-coordinate) based on a certain point. When encoding the position information, instead of directly encoding the coordinates of each 3D point, a method is used in which the position of each 3D point is expressed in an octree representation and the octree information is encoded to reduce the amount of code.

[0308] On the other hand, the attribute information includes information indicating color information (RGB, YUV, etc.) of each three-dimensional point, reflectance, normal vector, etc. For example, the three-dimensional data encoding device can encode the attribute information using an encoding method different from that for the position information.

[0309] In this embodiment, a method for encoding attribute information will be described. Note that in this embodiment, the attribute information values ​​will be described using integer values. For example, if each color component of RGB or YUV color information has 8-bit precision, each color component will take an integer value between 0 and 255. If the reflectance value has 10-bit precision, the reflectance value will take an integer value between 0 and 1023. Note that if the bit precision of the attribute information is decimal precision, the three-dimensional data encoding device may multiply the value by a scale value and then round it to an integer value so that the value of the attribute information becomes an integer value. Note that the three-dimensional data encoding device may add this scale value to the header of the bitstream, etc.

[0310] One possible method for encoding attribute information of a three-dimensional point is to calculate a predicted value of the attribute information of the three-dimensional point and encode the difference (prediction residual) between the value of the original attribute information and the predicted value. For example, if the value of the attribute information of a three-dimensional point p is Ap and the predicted value is Pp, the three-dimensional data encoding device encodes the absolute difference Diffp = |Ap - Pp|. In this case, if the predicted value Pp can be generated with high accuracy, the value of the absolute difference Diffp will be small. Therefore, for example, the amount of code can be reduced by entropy encoding the absolute difference Diffp using a coding table in which the smaller the value, the fewer bits are generated.

[0311] One method for generating predicted values ​​of attribute information is to use attribute information of a reference 3D point, which is another 3D point surrounding the target 3D point to be encoded. Here, the reference 3D point is a 3D point within a predetermined distance range from the target 3D point. For example, if there are target 3D point p = (x1, y1, z1) and 3D point q = (x2, y2, z2), the 3D data encoding device calculates the Euclidean distance d(p, q) between 3D point p and 3D point q as shown in (Equation A1).

[0312]

number

[0313] If the Euclidean distance d(p, q) is smaller than a predetermined threshold THd, the three-dimensional data encoding device determines that the position of the three-dimensional point q is close to the position of the target three-dimensional point p, and determines to use the value of the attribute information of the three-dimensional point q to generate a predicted value of the attribute information of the target three-dimensional point p. Note that other distance calculation methods may be used, such as Mahalanobis distance. The three-dimensional data encoding device may also determine that three-dimensional points outside a predetermined distance range from the target three-dimensional point are not used in the prediction process. For example, if a three-dimensional point r exists and the distance d(p, r) between the target three-dimensional point p and the three-dimensional point r is equal to or greater than a threshold THd, the three-dimensional data encoding device may determine that the three-dimensional point r is not used for prediction. Note that the three-dimensional data encoding device may add information indicating the threshold THd to a header of the bitstream, for example.

[0314] 51 is a diagram showing an example of a 3D point. In this example, the distance d(p, q) between the target 3D point p and the 3D point q is smaller than the threshold value THd. Therefore, the 3D data encoding device determines that the 3D point q is the reference 3D point of the target 3D point p, and determines that the value of the attribute information Aq of the 3D point q will be used to generate the predicted value Pp of the attribute information Ap of the target 3D point p.

[0315] On the other hand, the distance d(p, r) between the target 3D point p and the 3D point r is equal to or greater than the threshold value THd. Therefore, the 3D data encoding device determines that the 3D point r is not a reference 3D point of the target 3D point p, and determines not to use the value of the attribute information Ar of the 3D point r to generate the predicted value Pp of the attribute information Ap of the target 3D point p.

[0316] Furthermore, when the 3D data encoding device encodes attribute information of a target 3D point using a predicted value, it uses a 3D point whose attribute information has already been encoded and decoded as a reference 3D point. Similarly, when the 3D data decoding device decodes attribute information of a target 3D point to be decoded using a predicted value, it uses a 3D point whose attribute information has already been decoded as a reference 3D point. This allows the same predicted value to be generated during encoding and decoding, so that the bit stream of 3D points generated during encoding can be correctly decoded on the decoding side.

[0317] Furthermore, when encoding attribute information of 3D points, it is possible to classify each 3D point into multiple layers using the position information of the 3D points and then encode them. Here, each classified layer is called LoD (Level of Detail). The method of generating LoD will be explained using FIG. 52.

[0318] First, the three-dimensional data encoding device selects an initial point a0 and assigns it to LoD0. Next, the three-dimensional data encoding device extracts point a1 whose distance from point a0 is greater than the LoD0 threshold Thres_LoD[0] and assigns it to LoD0. Next, the three-dimensional data encoding device extracts point a2 whose distance from point a1 is greater than the LoD0 threshold Thres_LoD[0] and assigns it to LoD0. In this way, the three-dimensional data encoding device constructs LoD0 so that the distance between each point within LoD0 is greater than the threshold Thres_LoD[0].

[0319] Next, the three-dimensional data encoding device selects point b0, which has not yet been assigned an LoD, and assigns it to LoD1. Next, the three-dimensional data encoding device extracts point b1, whose distance from point b0 is greater than the LoD1 threshold Thres_LoD[1] and whose LoD has not been assigned, and assigns it to LoD1. Next, the three-dimensional data encoding device extracts point b2, whose distance from point b1 is greater than the LoD1 threshold Thres_LoD[1] and whose LoD has not been assigned, and assigns it to LoD1. In this way, the three-dimensional data encoding device configures LoD1 so that the distance between each point within LoD1 is greater than the threshold Thres_LoD[1].

[0320] Next, the three-dimensional data encoding device selects point c0, which has not yet been assigned an LoD, and assigns it to LoD2. Next, the three-dimensional data encoding device extracts point c1, which has an unassigned LoD and whose distance from point c0 is greater than the LoD2 threshold Thres_LoD[2], and assigns it to LoD2. Next, the three-dimensional data encoding device extracts point c2, which has an unassigned LoD and whose distance from point c1 is greater than the LoD2 threshold Thres_LoD[2], and assigns it to LoD2. In this way, the three-dimensional data encoding device configures LoD2 so that the distance between each point within LoD2 is greater than the threshold Thres_LoD[2]. For example, as shown in FIG. 53, thresholds Thres_LoD[0], Thres_LoD[1], and Thres_LoD[2] for each LoD are set.

[0321] Furthermore, the three-dimensional data encoding device may add information indicating the threshold values ​​of each LoD to the header of the bitstream, etc. For example, in the example shown in Fig. 53, the three-dimensional data encoding device may add threshold values ​​Thres_LoD[0], Thres_LoD[1], and Thres_LoD[2] to the header.

[0322] Alternatively, the three-dimensional data encoding device may assign all three-dimensional points to which no LoD has been assigned to the lowest layer of the LoD. In this case, the three-dimensional data encoding device can reduce the amount of header coding by not adding the threshold value for the lowest layer of the LoD to the header. For example, in the example shown in FIG. 53, the three-dimensional data encoding device adds thresholds Thres_LoD[0] and Thres_LoD[1] to the header, but does not add Thres_LoD[2] to the header. In this case, the three-dimensional data decoding device may estimate the value of Thres_LoD[2] to be 0. The three-dimensional data encoding device may also add the number of layers of the LoD to the header. This allows the three-dimensional data decoding device to determine the LoD of the lowest layer using the number of layers of the LoD.

[0323] In addition, by setting the threshold value of each LoD layer to be larger for higher layers as shown in Figure 53, the higher the layer (layer closer to LoD0), the sparser the point cloud becomes, with 3D points spaced farther apart, and the lower the layer, the denser the point cloud becomes, with 3D points spaced closer together. In the example shown in Figure 53, LoD0 is the top layer.

[0324] Furthermore, the method for selecting the initial 3D point when setting each LoD may depend on the encoding order used when encoding the position information. For example, the 3D data encoding device may select the 3D point that was encoded first when encoding the position information as the initial point a0 of LoD0, and then select points a1 and a2 using the initial point a0 as the base point to construct LoD0. The 3D data encoding device may then select, as the initial point b0 of LoD1, the 3D point whose position information is encoded earliest among the 3D points that do not belong to LoD0. In other words, the 3D data encoding device may select, as the initial point n0 of LoDn, the 3D point whose position information is encoded earliest among the 3D points that do not belong to the upper layers (LoD0 to LoDn-1) of LoDn. This allows the 3D data decoding device to use the same initial point selection method during decoding to construct the same LoD as during encoding, thereby enabling appropriate decoding of the bitstream. Specifically, the 3D data decoding device selects, as the initial point n0 of LoDn, the 3D point whose position information is decoded earliest among the 3D points that do not belong to the upper layers of LoDn.

[0325] A method for generating predicted values ​​of attribute information of 3D points using LoD information will be described below. For example, when encoding 3D points included in LoD0 in order, a 3D data encoding device generates a target 3D point included in LoD1 using coded and decoded attribute information (hereinafter simply referred to as "coded") included in LoD0 and LoD1. In this way, the 3D data encoding device generates predicted values ​​of attribute information of 3D points included in LoDn using coded attribute information included in LoDn' (n'<=n). In other words, the 3D data encoding device does not use attribute information of 3D points included in a layer below LoDn when calculating predicted values ​​of attribute information of 3D points included in LoDn.

[0326] For example, the three-dimensional data encoding device generates a predicted value of attribute information of a three-dimensional point by calculating the average of attribute values ​​of N or fewer three-dimensional points among encoded three-dimensional points surrounding the target three-dimensional point to be encoded. The three-dimensional data encoding device may also add the value of N to a bitstream header or the like. The three-dimensional data encoding device may also change the value of N for each three-dimensional point and add a value of N to each three-dimensional point. This allows an appropriate N to be selected for each three-dimensional point, thereby improving the accuracy of the predicted value. This reduces the prediction residual. The three-dimensional data encoding device may also add the value of N to the bitstream header and fix the value of N within the bitstream. This eliminates the need to encode or decode the value of N for each three-dimensional point, thereby reducing the amount of processing. The three-dimensional data encoding device may also encode the value of N separately for each LoD. This allows an appropriate N to be selected for each LoD, thereby improving coding efficiency.

[0327] Alternatively, the three-dimensional data encoding device may calculate a predicted value of the attribute information of a three-dimensional point as a weighted average value of the attribute information of N surrounding encoded three-dimensional points. For example, the three-dimensional data encoding device calculates the weight using distance information between the target three-dimensional point and each of the N surrounding three-dimensional points.

[0328] When a 3D data encoding device encodes the value of N separately for each LoD, it sets the value of N larger for higher LoD layers and smaller for lower LoD layers. Because the distance between 3D points in higher LoD layers is greater, it may be possible to improve prediction accuracy by setting the value of N larger and selecting and averaging multiple surrounding 3D points. Also, because the distance between 3D points in lower LoD layers is closer, it is possible to set the value of N smaller, reducing the amount of averaging processing and enabling efficient prediction.

[0329] Fig. 54 is a diagram showing an example of attribute information used for predicted values. As described above, the predicted value of point P included in LoDN is generated using the coded surrounding points P' included in LoDN' (N'<=N). Here, the surrounding points P' are selected based on the distance from point P. For example, the predicted value of the attribute information of point b2 shown in Fig. 54 is generated using the attribute information of points a0, a1, a2, b0, and b1.

[0330] The surrounding points selected vary depending on the value of N. For example, when N=5, a0, a1, a2, b0, and b1 are selected as surrounding points of point b2. When N=4, points a0, a1, a2, and b1 are selected based on distance information.

[0331] The predicted value is calculated by a distance-dependent weighted average. For example, in the example shown in FIG. 54, the predicted value a2p of point a2 is calculated by a weighted average of the attribute information of points a0 and a1, as shown in (Equation A2) and (Equation A3). i is the value of the attribute information of point ai.

[0332]

number

[0333] Furthermore, the predicted value b2p of point b2 is calculated by the weighted average of the attribute information of points a0, a1, a2, b0, and b1, as shown in (Equation A4) to (Equation A6). i is the value of the attribute information of point bi.

[0334]

number

[0335] Furthermore, the three-dimensional data encoding device may calculate a difference value (prediction residual) between the value of the attribute information of the three-dimensional point and a predicted value generated from the surrounding points, and quantize the calculated prediction residual. For example, the three-dimensional data encoding device performs quantization by dividing the prediction residual by a quantization scale (also called a quantization step). In this case, the smaller the quantization scale, the smaller the error (quantization error) that may occur due to quantization. Conversely, the larger the quantization scale, the larger the quantization error.

[0336] The three-dimensional data encoding device may change the quantization scale used for each LoD. For example, the three-dimensional data encoding device may use a smaller quantization scale for higher layers and a larger quantization scale for lower layers. Since the values ​​of attribute information of three-dimensional points belonging to higher layers may be used as predicted values ​​of attribute information of three-dimensional points belonging to lower layers, the encoding efficiency can be improved by reducing the quantization scale for higher layers to suppress quantization errors that may occur in higher layers and increasing the accuracy of predicted values. The three-dimensional data encoding device may add the quantization scale used for each LoD to a header or the like. This allows the three-dimensional data decoding device to correctly decode the quantization scale and therefore appropriately decode the bitstream.

[0337] Furthermore, the three-dimensional data encoding device may convert signed integer values ​​(signed quantized values), which are prediction residuals after quantization, into unsigned integer values ​​(unsigned quantized values). This eliminates the need to consider the occurrence of negative integers when entropy encoding the prediction residuals. Note that the three-dimensional data encoding device does not necessarily need to convert signed integer values ​​into unsigned integer values; for example, the sign bit may be separately entropy encoded.

[0338] The prediction residual is calculated by subtracting a predicted value from an original value. For example, the prediction residual a2r of point a2 is calculated by subtracting the predicted value a2p of point a2 from the attribute information value A2 of point a2, as shown in (Equation A7). The prediction residual b2r of point b2 is calculated by subtracting the predicted value b2p of point b2 from the attribute information value B2 of point b2, as shown in (Equation A8).

[0339] a2r=A2-a2p (Equation A7) b2r=B2-b2p (Equation A8)

[0340] Furthermore, the prediction residual is quantized by dividing it by QS (Quantization Step). For example, the quantized value a2q of point a2 is calculated by (Equation A9). The quantized value b2q of point b2 is calculated by (Equation A10). Here, QS_LoD0 is the QS for LoD0, and QS_LoD1 is the QS for LoD1. That is, the QS may be changed depending on the LoD.

[0341] a2q=a2r / QS_LoD0 (Equation A9) b2q=b2r / QS_LoD1 (Formula A10)

[0342] Furthermore, the three-dimensional data encoding device converts the signed integer value, which is the quantized value, into an unsigned integer value as follows: If the signed integer value a2q is less than 0, the three-dimensional data encoding device sets the unsigned integer value a2u to -1-(2×a2q). If the signed integer value a2q is 0 or greater, the three-dimensional data encoding device sets the unsigned integer value a2u to 2×a2q.

[0343] Similarly, the three-dimensional data encoding device sets the unsigned integer value b2u to -1-(2×b2q) if the signed integer value b2q is less than 0. The three-dimensional data encoding device sets the unsigned integer value b2u to 2×b2q if the signed integer value b2q is greater than or equal to 0.

[0344] Furthermore, the three-dimensional data encoding device may encode the quantized prediction residuals (unsigned integer values) by entropy encoding. For example, the unsigned integer values ​​may be binarized and then subjected to binary arithmetic encoding.

[0345] In this case, the three-dimensional data encoding device may switch the binarization method depending on the value of the prediction residual. For example, when the prediction residual pu is smaller than a threshold R_TH, the three-dimensional data encoding device binarizes the prediction residual pu using a fixed number of bits required to express the threshold R_TH. When the prediction residual pu is equal to or greater than the threshold R_TH, the three-dimensional data encoding device binarizes the binarized data of the threshold R_TH and the value (pu-R_TH) using Exponential-Golomb or the like.

[0346] For example, if the threshold value R_TH is 63 and the prediction residual pu is smaller than 63, the three-dimensional data encoding device binarizes the prediction residual pu using 6 bits. Furthermore, if the prediction residual pu is 63 or greater, the three-dimensional data encoding device performs arithmetic encoding by binarizing the binary data of the threshold value R_TH (111111) and (pu-63) using Exponential Golomb algorithm.

[0347] In a more specific example, when the prediction residual pu is 32, the three-dimensional data encoding device generates 6-bit binary data (100000) and arithmetically encodes this bit string. Also, when the prediction residual pu is 66, the three-dimensional data encoding device generates binary data (111111) of the threshold R_TH and a bit string (00100) that expresses the value 3 (66-63) in Exponential Golomb notation, and arithmetically encodes this bit string (111111+00100).

[0348] In this way, the 3D data encoding device switches the binarization method depending on the size of the prediction residual, thereby enabling encoding while suppressing a sudden increase in the number of binarization bits when the prediction residual becomes large. Note that the 3D data encoding device may add the threshold value R_TH to the header of the bitstream, etc.

[0349] For example, when encoding is performed at a high bit rate, that is, when the quantization scale is small, the quantization error is small and the prediction accuracy is high, and as a result, the prediction residual may not be large. Therefore, in this case, the three-dimensional data encoding device sets the threshold R_TH to a large value. This reduces the possibility of encoding binarized data of the threshold R_TH, and improves encoding efficiency. Conversely, when encoding is performed at a low bit rate, that is, when the quantization scale is large, the quantization error is large and the prediction accuracy is poor, and as a result, the prediction residual may be large. Therefore, in this case, the three-dimensional data encoding device sets the threshold R_TH to a small value. This prevents a sudden increase in the bit length of the binarized data.

[0350] Furthermore, the three-dimensional data encoding device may switch the threshold R_TH for each LoD and add the threshold R_TH for each LoD to a header or the like. That is, the three-dimensional data encoding device may switch the binarization method for each LoD. For example, in higher layers, the distance between three-dimensional points is large, which may result in poor prediction accuracy and large prediction residuals. Therefore, the three-dimensional data encoding device prevents a sudden increase in the bit length of the binarized data by setting a small threshold R_TH for higher layers. Furthermore, in lower layers, the distance between three-dimensional points is small, which may result in high prediction accuracy and small prediction residuals. Therefore, the three-dimensional data encoding device improves encoding efficiency by setting a large threshold R_TH for each layer.

[0351] Fig. 55 is a diagram showing an example of exponential-Golomb coding, illustrating the relationship between values ​​(multiple values) before binarization and bits (codes) after binarization. Note that 0 and 1 shown in Fig. 55 may be reversed.

[0352] Furthermore, the three-dimensional data encoding device applies arithmetic coding to the binary data of the prediction residual, thereby improving encoding efficiency. Note that when applying arithmetic coding, the tendency of the occurrence probability of 0 and 1 for each bit may differ between the n-bit code, which is the portion of the binary data binarized with n bits, and the remaining code, which is the portion binarized using the Exponential Golomb algorithm. Therefore, the three-dimensional data encoding device may switch the application method of arithmetic coding between the n-bit code and the remaining code.

[0353] For example, a three-dimensional data encoder performs arithmetic coding on an n-bit code using a different coding table (probability table) for each bit. In this case, the three-dimensional data encoder may change the number of coding tables used for each bit. For example, the three-dimensional data encoder performs arithmetic coding on the first bit b0 of an n-bit code using one coding table. The three-dimensional data encoder also uses two coding tables for the next bit b1. The three-dimensional data encoder also switches the coding table used for arithmetic coding of bit b1 depending on the value of b0 (0 or 1). Similarly, the three-dimensional data encoder uses four coding tables for the next bit b2. The three-dimensional data encoder also switches the coding table used for arithmetic coding of bit b2 depending on the values ​​of b0 and b1 (0 to 3).

[0354] In this way, the three-dimensional data encoding device performs arithmetic encoding on each bit bn-1 of the n-bit code. n-1 The three-dimensional data encoding device uses coding tables. In addition, the three-dimensional data encoding device switches the coding table to be used depending on the value (occurrence pattern) of the bits before bn-1. This allows the three-dimensional data encoding device to use an appropriate coding table for each bit, thereby improving coding efficiency.

[0355] Note that the three-dimensional data encoding device may reduce the number of encoding tables used for each bit. For example, when arithmetic-encoding each bit bn-1, the three-dimensional data encoding device may switch between two encoding tables according to the value (occurrence pattern) of the m bits (m < n-1) before bn-1. This can improve the encoding efficiency while suppressing the number of encoding tables used for each bit. Note that the three-dimensional data encoding device may update the occurrence probabilities of 0 and 1 in each encoding table according to the value of the actually-occurred binarized data. Also, the three-dimensional data encoding device may fix the occurrence probabilities of 0 and 1 in the encoding tables for some bits. This can suppress the number of updates of the occurrence probabilities, thus reducing the processing amount. m For example, when the n-bit code is b0b1b2…bn-1, the encoding table for b0 is one (CTb0). The encoding table for b1 is two (CTb10, CTb11). Also, the encoding table to be used is switched according to the value (0 to 1) of b0. The encoding table for b2 is four (CTb20, CTb21, CTb22, CTb23). Also, the encoding table to be used is switched according to the values (0 to 3) of b0 and b1. The encoding table for bn-1 is two

[0356] n-1 n-1 n-1 n n

[0357] <0 / / (CTbn0, CTbn1, …, CTbn(2

[0358] -1)) and the encoding table to be used is switched according to the value (0 to 2 n -1) of b0b1…bn-2. Note that the three-dimensional data encoding device may apply m-ary arithmetic encoding (m = 2<0000 / / (CTbn0, CTbn1, …, CTbn(2<0001 / / (CTbn0, CTbn1, …, CTbn(2Fig. 56 is a diagram for explaining processing when the residual code is an exponential-Golomb code, for example. The residual code, which is the portion binarized using the exponential-Golomb code, includes a prefix portion and a suffix portion, as shown in Fig. 56. For example, the three-dimensional data encoding device switches encoding tables between the prefix portion and the suffix portion. That is, the three-dimensional data encoding device arithmetically encodes each bit included in the prefix portion using the encoding table for the prefix, and arithmetically encodes each bit included in the suffix portion using the encoding table for the suffix.

[0359] The three-dimensional data encoding device may update the occurrence probabilities of 0 and 1 in each encoding table according to the value of the binary data that actually occurs. Alternatively, the three-dimensional data encoding device may fix the occurrence probabilities of 0 and 1 in one of the encoding tables. This reduces the number of updates to the occurrence probabilities, thereby reducing the amount of processing. For example, the three-dimensional data encoding device may update the occurrence probabilities for the prefix part and fix the occurrence probabilities for the suffix part.

[0360] Furthermore, the three-dimensional data encoding device decodes the quantized prediction residual by inverse quantization and reconstruction, and uses the decoded value, which is the decoded prediction residual, for predicting the three-dimensional point to be encoded and beyond. Specifically, the three-dimensional data encoding device calculates an inverse quantized value by multiplying the quantized prediction residual (quantized value) by the quantization scale, and obtains a decoded value (reconstructed value) by adding the inverse quantized value and the predicted value.

[0361] For example, the inverse quantization value a2iq of point a2 is calculated by (Equation A11) using the quantization value a2q of point a2. The inverse quantization value b2iq of point b2 is calculated by (Equation A12) using the quantization value b2q of point b2. Here, QS_LoD0 is the QS for LoD0, and QS_LoD1 is the QS for LoD1. In other words, the QS may change depending on the LoD.

[0362] a2iq=a2q×QS_LoD0 (Formula A11) b2iq=b2q×QS_LoD1 (Formula A12)

[0363] For example, the decoded value a2rec of point a2 is calculated by adding the predicted value a2p of point a2 to the inverse quantized value a2iq of point a2 as shown in (Equation A13). The decoded value b2rec of point b2 is calculated by adding the predicted value b2p of point b2 to the inverse quantized value b2iq of point b2 as shown in (Equation A14).

[0364] a2rec=a2iq+a2p (formula A13) b2rec=b2iq+b2p (formula A14)

[0365] An example of the syntax of a bitstream according to this embodiment will be described below. Fig. 57 is a diagram showing an example of the syntax of an attribute header (attribute_header) according to this embodiment. The attribute header is header information of attribute information. As shown in Fig. 57, the attribute header includes hierarchical number information (NumLoD), three-dimensional point number information (NumOfPoint[i]), hierarchical threshold (Thres_Lod[i]), neighboring point number information (NumNeighorPoint[i]), predicted threshold (THd[i]), quantization scale (QS[i]), and binarization threshold (R_TH[i]).

[0366] The number of layers information (NumLoD) indicates the number of layers of the LoD to be used.

[0367] The three-dimensional point number information (NumOfPoint[i]) indicates the number of three-dimensional points belonging to layer i. The three-dimensional data encoding device may add three-dimensional point total number information (AllNumOfPoint), indicating the total number of three-dimensional points, to a separate header. In this case, the three-dimensional data encoding device does not need to add NumOfPoint[NumLoD-1], indicating the number of three-dimensional points belonging to the lowest layer, to the header. In this case, the three-dimensional data decoding device can calculate NumOfPoint[NumLoD-1] using (Equation A15). This allows the amount of coding in the header to be reduced.

[0368] [Number]

[0369] The hierarchical threshold (Thres_Lod[i]) is the threshold used for the setting of layer i. The three-dimensional data encoding device and the three-dimensional data decoding device configure LoDi such that the distance between each point in LoDi is greater than the threshold Thres_LoD[i]. Also, the three-dimensional data encoding device may not add the value of Thres_Lod[NumLoD - 1] (the lowest layer) to the header. In this case, the three-dimensional data decoding device estimates the value of Thres_Lod[NumLoD - 1] as 0. Thereby, the amount of code for the header can be reduced.

[0370] The number of surrounding points information (NumNeighorPoint[i]) indicates the upper limit value of the number of surrounding points used for generating the predicted value of the three-dimensional points belonging to layer i. When the number of surrounding points M is less than NumNeighorPoint[i] (M < NumNeighorPoint[i]), the three-dimensional data encoding device may calculate the predicted value using M surrounding points. Also, when the three-dimensional data encoding device does not need to divide the value of NumNeighorPoint[i] for each LoD, it may add one piece of surrounding points information (NumNeighorPoint) used for all LoDs to the header.

[0371] The prediction threshold (THd[i]) indicates the upper limit value of the distance between the surrounding three-dimensional points and the target three-dimensional point used for predicting the target three-dimensional point to be encoded or decoded at layer i. The three-dimensional data encoding device and the three-dimensional data decoding device do not use the three-dimensional points whose distance from the target three-dimensional point is farther than THd[i] for prediction. Note that when the three-dimensional data encoding device does not need to divide the value of THd[i] for each LoD, it may add one prediction threshold (THd) used for all LoDs to the header.

[0372] The quantization scale (QS[i]) indicates the quantization scale used for quantization and inverse quantization of layer i.

[0373] The binarization threshold (R_TH[i]) is a threshold for switching the binarization method for the prediction residual of a 3D point belonging to layer i. For example, if the prediction residual is smaller than the threshold R_TH, the 3D data encoding device binarizes the prediction residual pu with a fixed number of bits, and if the prediction residual is equal to or greater than the threshold R_TH, the 3D data encoding device binarizes the binarized data of the threshold R_TH and the value (pu-R_TH) using the Exponential Golomb algorithm. Note that if it is not necessary to switch the value of R_TH[i] for each LoD, the 3D data encoding device may add a single binarization threshold (R_TH) used for all LoDs to the header.

[0374] Note that R_TH[i] may be the maximum value that can be represented in n bits. For example, R_TH is 63 in 6 bits, and R_TH is 255 in 8 bits. The three-dimensional data encoding device may also encode the number of bits as the binarization threshold instead of encoding the maximum value that can be represented in n bits. For example, the three-dimensional data encoding device may add a value of 6 to the header when R_TH[i]=63, or a value of 8 to the header when R_TH[i]=255. The three-dimensional data encoding device may also define a minimum value (minimum number of bits) for the number of bits representing R_TH[i], and add the relative number of bits from the minimum value to the header. For example, the three-dimensional data encoding device may add a value of 0 to the header when R_TH[i]=63 and the minimum number of bits is 6, or add a value of 2 to the header when R_TH[i]=255 and the minimum number of bits is 6.

[0375] The three-dimensional data encoding device may also entropy-encode at least one of NumLod, Thres_Lod[i], NumNeighborPoint[i], THd[i], QS[i], and R_TH[i] and add the result to the header. For example, the three-dimensional data encoding device may binarize each value and arithmetically encode it. The three-dimensional data encoding device may also encode each value at a fixed length to reduce the amount of processing.

[0376] Furthermore, the three-dimensional data encoding device may not need to add at least one of NumLod, Thres_Lod[i], NumNeighborPoint[i], THd[i], QS[i], and R_TH[i] to the header. For example, the value of at least one of these may be specified by a profile or level of a standard, etc. This allows the number of bits in the header to be reduced.

[0377] Fig. 58 is a diagram showing an example of the syntax of attribute data (attribute_data) according to this embodiment. This attribute data includes coded data of attribute information of multiple three-dimensional points. As shown in Fig. 58, the attribute data includes an n-bit code and a remaining code.

[0378] An n-bit code is the coded data of the prediction residual of the attribute information value or a part thereof. The bit length of the n-bit code depends on the value of R_TH[i]. For example, if the value indicated by R_TH[i] is 63, the n-bit code is 6 bits, and if the value indicated by R_TH[i] is 255, the n-bit code is 8 bits.

[0379] The remaining code is the coded data of the prediction residual of the attribute information value that is coded using the exponential Golomb coding method. This remaining code is coded or decoded when the n-bit code is the same as R_TH[i]. The three-dimensional data decoding device also decodes the prediction residual by adding the value of the n-bit code and the value of the remaining code. Note that if the n-bit code is not the same value as R_TH[i], the remaining code does not need to be coded or decoded.

[0380] The flow of processing in the three-dimensional data encoding device will be explained below. Figure 59 is a flowchart of three-dimensional data encoding processing by the three-dimensional data encoding device.

[0381] First, the three-dimensional data encoding device encodes position information (geometry) (S3001). For example, the three-dimensional data is encoded using an octree representation.

[0382] If the position of a three-dimensional point changes due to quantization or the like after encoding the position information, the three-dimensional data encoding device reallocates attribute information of the original three-dimensional point to the changed three-dimensional point (S3002). For example, the three-dimensional data encoding device performs the reallocation by interpolating the value of the attribute information according to the amount of change in position. For example, the three-dimensional data encoding device detects N pre-change three-dimensional points that are close to the changed three-dimensional position, and calculates a weighted average of the attribute information values ​​of the N three-dimensional points. For example, the three-dimensional data encoding device determines a weight in the weighted average based on the distance from the changed three-dimensional position to each of the N three-dimensional points. Then, the three-dimensional data encoding device determines the value obtained by the weighted average as the value of the attribute information of the changed three-dimensional point. Furthermore, if two or more three-dimensional points change to the same three-dimensional position due to quantization or the like, the three-dimensional data encoding device may assign the average value of the attribute information of the two or more pre-change three-dimensional points as the value of the attribute information of the changed three-dimensional point.

[0383] Next, the three-dimensional data encoding device encodes the reallocated attribute information (Attribute) (S3003). For example, when encoding multiple types of attribute information, the three-dimensional data encoding device may encode the multiple types of attribute information in order. For example, when encoding color and reflectance as attribute information, the three-dimensional data encoding device may generate a bitstream in which the encoding result of reflectance is added after the encoding result of color. Note that the order in which the encoding results of the multiple attribute information are added to the bitstream is not limited to this order and may be any order.

[0384] The three-dimensional data encoding device may also add information indicating the start location of the encoded data for each piece of attribute information in the bitstream to a header or the like. This allows the three-dimensional data decoding device to selectively decode attribute information that needs to be decoded, thereby omitting the decoding process for attribute information that does not need to be decoded. This reduces the amount of processing required by the three-dimensional data decoding device. The three-dimensional data encoding device may also encode multiple types of attribute information in parallel and combine the encoding results into a single bitstream. This allows the three-dimensional data encoding device to encode multiple types of attribute information at high speed.

[0385] 60 is a flowchart of the attribute information encoding process (S3003). First, the three-dimensional data encoding device sets the LoD (S3011). That is, the three-dimensional data encoding device assigns each three-dimensional point to one of a plurality of LoDs.

[0386] Next, the three-dimensional data encoding device starts a loop for each LoD (S3012). That is, the three-dimensional data encoding device repeatedly performs the processes of steps S3013 to S3021 for each LoD.

[0387] Next, the three-dimensional data encoding device starts a loop for each three-dimensional point (S3013). That is, the three-dimensional data encoding device repeatedly performs the processes of steps S3014 to S3020 for each three-dimensional point.

[0388] First, the three-dimensional data encoding device searches for a plurality of surrounding points, which are three-dimensional points existing around the target three-dimensional point to be processed, to be used in calculating a predicted value of the target three-dimensional point (S3014). Next, the three-dimensional data encoding device calculates a weighted average of the values ​​of the attribute information of the plurality of surrounding points, and sets the obtained value as the predicted value P (S3015). Next, the three-dimensional data encoding device calculates a prediction residual, which is the difference between the attribute information of the target three-dimensional point and the predicted value (S3016). Next, the three-dimensional data encoding device calculates a quantized value by quantizing the prediction residual (S3017). Next, the three-dimensional data encoding device arithmetically encodes the quantized value (S3018).

[0389] The three-dimensional data encoding device also calculates inverse quantized values ​​by inverse quantizing the quantized values ​​(S3019). Next, the three-dimensional data encoding device generates decoded values ​​by adding predicted values ​​to the inverse quantized values ​​(S3020). Next, the three-dimensional data encoding device ends the loop in units of three-dimensional points (S3021). Also, the three-dimensional data encoding device ends the loop in units of LoDs (S3022).

[0390] Hereinafter, a three-dimensional data decoding process in a three-dimensional data decoding device that decodes a bit stream generated by the above three-dimensional data encoding device will be described.

[0391] The three-dimensional data decoding device generates decoded binary data by arithmetically decoding the binary data of the attribute information in the bit stream generated by the three-dimensional data encoding device using the same method as the three-dimensional data encoding device. Note that, in the three-dimensional data encoding device, if the application method of arithmetic coding is switched between the part binarized with n bits (n-bit code) and the part binarized using Exponential Golomb (remaining code), the three-dimensional data decoding device performs decoding accordingly when applying arithmetic decoding.

[0392] For example, in an arithmetic decoding method for an n-bit code, a three-dimensional data decoding device performs arithmetic decoding using a different coding table (decoding table) for each bit. At this time, the three-dimensional data decoding device may change the number of coding tables used for each bit. For example, arithmetic decoding is performed using one coding table for the first bit b0 of an n-bit code. The three-dimensional data decoding device also uses two coding tables for the next bit b1. The three-dimensional data decoding device also switches the coding table used for arithmetic decoding of bit b1 depending on the value of b0 (0 or 1). Similarly, the three-dimensional data decoding device uses four coding tables for the next bit b2. The three-dimensional data decoding device also switches the coding table used for arithmetic decoding of bit b2 depending on the values ​​of b0 and b1 (0 to 3).

[0393] Thus, when the three-dimensional data decoder arithmetically decodes each bit bn-1 of the n-bit code, it uses n-1 two encoding tables. Also, the three-dimensional data decoder switches the encoding table to be used according to the values (generated patterns) of the bits before bn-1. Thereby, the three-dimensional data decoder can appropriately decode a bit stream with improved encoding efficiency by using an appropriate encoding table for each bit.

[0394] Note that the three-dimensional data decoder may reduce the number of encoding tables used for each bit. For example, when the three-dimensional data decoder arithmetically decodes each bit bn-1, it may switch two m encoding tables according to the values (generated patterns) of the m bits (m < n-1) before bn-1. Thereby, the three-dimensional data decoder can appropriately decode a bit stream with improved encoding efficiency while suppressing the number of encoding tables used for each bit. Note that the three-dimensional data decoder may update the occurrence probabilities of 0 and 1 in each encoding table according to the values of the actually generated binarized data. Also, the three-dimensional data decoder may fix the occurrence probabilities of 0 and 1 in the encoding tables of some bits. Thereby, the number of updates of the occurrence probabilities can be suppressed, so the processing amount can be reduced.

[0395] For example, when the n-bit code is b0b1b2…bn-1, the encoding table for b0 is one (CTb0). The encoding tables for b1 are two (CTb10, CTb11). Also, the encoding table is switched according to the value (0 to 1) of b0. The encoding tables for b2 are four (CTb20, CTb21, CTb22, CTb23). Also, the encoding table is switched according to the values (0 to 3) of b0 and b1. The encoding tables for bn-1 are n-1 two (CTbn0, CTbn1, …, CTbn(2 n-1 -1)). Also, the encoding table is switched according to the values (0 to 2 n-1 -1) of b0b1…bn-2.

[0396] Fig. 61 is a diagram for explaining processing when, for example, the residual code is an exponential-Golomb code. The portion (residual code) that the three-dimensional data encoding device binarizes and encodes using the exponential-Golomb code includes a prefix portion and a suffix portion, as shown in Fig. 61. For example, the three-dimensional data decoding device switches the encoding table between the prefix portion and the suffix portion. That is, the three-dimensional data decoding device arithmetically decodes each bit included in the prefix portion using the encoding table for the prefix, and arithmetically decodes each bit included in the suffix portion using the encoding table for the suffix.

[0397] The three-dimensional data decoding device may update the occurrence probabilities of 0 and 1 in each encoding table according to the value of the binarized data generated during decoding. Alternatively, the three-dimensional data decoding device may fix the occurrence probabilities of 0 and 1 in one of the encoding tables. This reduces the number of updates to the occurrence probabilities, thereby reducing the amount of processing. For example, the three-dimensional data decoding device may update the occurrence probabilities for the prefix part and fix the occurrence probabilities for the suffix part.

[0398] Furthermore, the three-dimensional data decoding device decodes the quantized prediction residual (unsigned integer value) by multi-valuing the binary data of the arithmetically decoded prediction residual in accordance with the encoding method used in the three-dimensional data encoding device. The three-dimensional data decoding device first calculates the value of the decoded n-bit code by arithmetically decoding the binary data of the n-bit code. Next, the three-dimensional data decoding device compares the value of the n-bit code with the value of R_TH.

[0399] When the value of the n-bit code matches the value of R_TH, the three-dimensional data decoding device determines that a bit coded using the exponential-Golomb method exists next, and arithmetically decodes the remaining code, which is binary data coded using the exponential-Golomb method.The three-dimensional data decoding device then calculates the value of the remaining code from the decoded remaining code using a reverse lookup table showing the relationship between the remaining code and its value. FIG. 62 is a diagram showing an example of a reverse lookup table showing the relationship between the remaining code and its value.The three-dimensional data decoding device then adds the value of the obtained remaining code to R_TH to obtain a multi-valued post-quantization prediction residual.

[0400] On the other hand, if the value of the n-bit code does not match the value of R_TH (the value is smaller than R_TH), the three-dimensional data decoding device determines the value of the n-bit code as the multi-valued post-quantization prediction residual. This allows the three-dimensional data decoding device to properly decode the bitstream generated by the three-dimensional data encoding device by switching the binarization method according to the value of the prediction residual.

[0401] If the threshold value R_TH is added to the header or the like of the bitstream, the three-dimensional data decoding device may decode the value of the threshold value R_TH from the header and switch the decoding method using the decoded threshold value R_TH. Also, if the threshold value R_TH is added to the header or the like for each LoD, the three-dimensional data decoding device switches the decoding method using the decoded threshold value R_TH for each LoD.

[0402] For example, if the threshold R_TH is 63 and the value of the decoded n-bit code is 63, the three-dimensional data decoding device obtains the value of the residual code by decoding the residual code using the Exponential Golomb method. For example, in the example shown in FIG. 62, the residual code is 00100, and the value of the residual code is obtained as 3. Next, the three-dimensional data decoding device obtains the value of the prediction residual, 66, by adding the value of the threshold R_TH, 63, and the value of the residual code, 3.

[0403] Furthermore, if the value of the decoded n-bit code is 32, the three-dimensional data decoding device sets the value of the n-bit code, 32, as the value of the prediction residual.

[0404] Furthermore, the three-dimensional data decoding device converts the decoded quantized prediction residuals from unsigned integer values ​​to signed integer values, for example, by a process reverse to that performed by the three-dimensional data encoding device. This allows the three-dimensional data decoding device to appropriately decode the generated bitstream when entropy encoding the prediction residuals, without taking into account the occurrence of negative integers. Note that the three-dimensional data decoding device does not necessarily need to convert unsigned integer values ​​to signed integer values; for example, when decoding a bitstream generated by separately entropy encoding the sign bits, the sign bits may be decoded.

[0405] The three-dimensional data decoding device generates decoded values ​​by decoding the quantized prediction residuals converted into signed integer values ​​through inverse quantization and reconstruction. The three-dimensional data decoding device also uses the generated decoded values ​​for prediction of the three-dimensional point to be decoded and thereafter. Specifically, the three-dimensional data decoding device calculates an inverse quantization value by multiplying the quantized prediction residuals by the decoded quantization scale, and obtains a decoded value by adding the inverse quantization value and the prediction value.

[0406] The decoded unsigned integer value (unsigned quantized value) is converted into a signed integer value by the following process. If the LSB (least significant bit) of the decoded unsigned integer value a2u is 1, the three-dimensional data decoding device sets the signed integer value a2q to -((a2u+1)>>1). If the LSB of the unsigned integer value a2u is not 1, the three-dimensional data decoding device sets the signed integer value a2q to (a2u>>1).

[0407] Similarly, if the LSB of the decoded unsigned integer value b2u is 1, the three-dimensional data decoding device sets the signed integer value b2q to -((b2u+1)>>1). If the LSB of the unsigned integer value n2u is not 1, the three-dimensional data decoding device sets the signed integer value b2q to (b2u>>1).

[0408] Furthermore, the details of the inverse quantization and reconstruction processing by the three-dimensional data decoding device are the same as those of the inverse quantization and reconstruction processing by the three-dimensional data encoding device.

[0409] The flow of processing in the three-dimensional data decoding device will be explained below. Figure 63 is a flowchart of three-dimensional data decoding processing by the three-dimensional data decoding device. First, the three-dimensional data decoding device decodes position information (geometry) from the bitstream (S3031). For example, the three-dimensional data decoding device performs decoding using an octree representation.

[0410] Next, the three-dimensional data decoding device decodes attribute information (Attribute) from the bitstream (S3032). For example, when decoding multiple types of attribute information, the three-dimensional data decoding device may decode the multiple types of attribute information in order. For example, when decoding color and reflectance as attribute information, the three-dimensional data decoding device decodes the encoding result of color and the encoding result of reflectance in the order in which they are added to the bitstream. For example, when the encoding result of reflectance is added after the encoding result of color in the bitstream, the three-dimensional data decoding device decodes the encoding result of color, and then decodes the encoding result of reflectance. Note that the three-dimensional data decoding device may decode the encoding results of attribute information added to the bitstream in any order.

[0411] Furthermore, the three-dimensional data decoding device may obtain information indicating the start location of the encoded data for each piece of attribute information in the bitstream by decoding a header or the like. This allows the three-dimensional data decoding device to selectively decode attribute information that requires decoding, thereby omitting the decoding process for attribute information that does not require decoding. This reduces the amount of processing required by the three-dimensional data decoding device. Furthermore, the three-dimensional data decoding device may decode multiple types of attribute information in parallel and integrate the decoding results into a single three-dimensional point cloud. This allows the three-dimensional data decoding device to decode multiple types of attribute information at high speed.

[0412] 64 is a flowchart of the attribute information decoding process (S3032). First, the three-dimensional data decoding device sets the LoD (S3041). That is, the three-dimensional data decoding device assigns each of the multiple three-dimensional points having decoded position information to one of the multiple LoDs. For example, this assignment method is the same as the assignment method used in the three-dimensional data encoding device.

[0413] Next, the three-dimensional data decoding device starts a loop for each LoD (S3042). That is, the three-dimensional data decoding device repeatedly performs the processes of steps S3043 to S3049 for each LoD.

[0414] Next, the three-dimensional data decoding device starts a loop for each three-dimensional point (S3043). That is, the three-dimensional data decoding device repeatedly performs the processes of steps S3044 to S3048 for each three-dimensional point.

[0415] First, the three-dimensional data decoding device searches for a plurality of surrounding points, which are three-dimensional points existing around the target three-dimensional point to be processed, and are used to calculate a predicted value of the target three-dimensional point (S3044). Next, the three-dimensional data decoding device calculates a weighted average of the values ​​of the attribute information of the plurality of surrounding points, and sets the obtained value as the predicted value P (S3045). Note that these processes are similar to those in the three-dimensional data encoding device.

[0416] Next, the three-dimensional data decoding device arithmetically decodes the quantized value from the bitstream (S3046). The three-dimensional data decoding device also calculates an inverse quantized value by inverse quantizing the decoded quantized value (S3047). Next, the three-dimensional data decoding device generates a decoded value by adding a predicted value to the inverse quantized value (S3048). Next, the three-dimensional data decoding device ends the loop in three-dimensional point units (S3049). The three-dimensional data decoding device also ends the loop in LoD units (S3050).

[0417] Next, the configurations of a three-dimensional data encoding device and a three-dimensional data decoding device according to this embodiment will be described. Fig. 65 is a block diagram showing the configuration of a three-dimensional data encoding device 3000 according to this embodiment. This three-dimensional data encoding device 3000 includes a position information encoding unit 3001, an attribute information reallocation unit 3002, and an attribute information encoding unit 3003.

[0418] The attribute information encoding unit 3003 encodes position information (geometry) of multiple 3D points included in the input point cloud. The attribute information reallocation unit 3002 reallocates values ​​of attribute information of multiple 3D points included in the input point cloud using the results of encoding and decoding the position information. The attribute information encoding unit 3003 encodes the reallocated attribute information. In addition, the 3D data encoding device 3000 generates a bitstream including the encoded position information and the encoded attribute information.

[0419] 66 is a block diagram showing the configuration of a three-dimensional data decoding device 3010 according to this embodiment. This three-dimensional data decoding device 3010 includes a position information decoding unit 3011 and an attribute information decoding unit 3012.

[0420] The position information decoding unit 3011 decodes position information (geometry) of multiple 3D points from the bitstream. The attribute information decoding unit 3012 decodes attribute information (attribute) of multiple 3D points from the bitstream. The 3D data decoding device 3010 generates an output point cloud by combining the decoded position information and the decoded attribute information.

[0421] (Embodiment 5) Hereinafter, the 3D point to be coded may be referred to as a first 3D point, and the 3D points around it may be referred to as second 3D points.

[0422] For example, when generating a predicted value of attribute information of a 3D point, the attribute value of the 3D point that is closest to the 3D point to be encoded and decoded may be generated as is as the predicted value. Furthermore, when generating a predicted value, prediction mode information (PredMode) may be added to each 3D point, and a predicted value may be generated by selecting one predicted value from multiple predicted values. That is, for a total of M prediction modes, for example, the average value may be assigned to prediction mode 0, the attribute value of 3D point A to prediction mode 1, ..., the attribute value of 3D point Z to prediction mode M-1, and the prediction mode used for the prediction may be added to the bitstream for each 3D point. In this way, the first prediction mode value indicating the first prediction mode in which the average of the attribute information of the surrounding 3D points is calculated as the predicted value may be smaller than the second prediction mode value indicating the second prediction mode in which the attribute information of the surrounding 3D points itself is calculated as the predicted value. Here, the "average value" that is the predicted value calculated in prediction mode 0 is the average value of the attribute values ​​of the 3D points surrounding the 3D point to be encoded.

[0423] Fig. 67 is a diagram showing a first example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. Fig. 68 is a diagram showing an example of attribute information used for predicted values ​​according to Embodiment 5. Fig. 69 is a diagram showing a second example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5.

[0424] The number of prediction modes M may be added to the bitstream. Alternatively, the number of prediction modes M may not be added to the bitstream, but may have a value defined by a profile, level, or the like of a standard. Alternatively, the number of prediction modes M may use a value calculated from the number of three-dimensional points N used for prediction. For example, the number of prediction modes M may be calculated by M=N+1.

[0425] Note that the table shown in FIG. 67 is an example in which the number of three-dimensional points used for prediction N=4 and the number of prediction modes M=5. A predicted value of the attribute information of point b2 can be generated using the attribute information of points a0, a1, a2, and b1. When selecting one prediction mode from a plurality of prediction modes, a prediction mode that generates the attribute values ​​of each of points a0, a1, a2, and b1 as predicted values ​​may be selected based on distance information from point b2 to each of points a0, a1, a2, and b1. A prediction mode is assigned to each three-dimensional point to be coded. The predicted value is calculated according to a value corresponding to the assigned prediction mode.

[0426] The table shown in FIG. 69 is an example in which the number of three-dimensional points used for prediction, N=4, and the number of prediction modes, M=5, as in FIG. 67. A predicted value of the attribute information of point a2 can be generated using the attribute information of points a0 and a1. When selecting one prediction mode from a plurality of prediction modes, a prediction mode that generates the attribute values ​​of each of points a0 and a1 as predicted values ​​may be selected based on distance information from point a2 to each of points a0 and a1. A prediction mode is assigned to each three-dimensional point to be coded. The predicted value is calculated according to a value corresponding to the assigned prediction mode.

[0427] Note that when the number of adjacent points, that is, the number of surrounding three-dimensional points N, is less than four, as in the case of point a2, prediction modes to which prediction values ​​are not assigned in the table may be set as "not available."

[0428] Note that the assignment of prediction mode values ​​may be determined in order of distance from the 3D point to be coded. For example, the prediction mode values ​​indicating multiple prediction modes are smaller as the distance from the 3D point to be coded to surrounding 3D points having attribute information used as predicted values ​​decreases. In the example of FIG. 67, the order of points b1, a2, a1, and a0 indicates that the distance to point b2, which is the 3D point to be coded, decreases. For example, in calculating predicted values, attribute information of point b1 is calculated as a predicted value in a prediction mode indicated by a prediction mode value of "1" among two or more prediction modes, and attribute information of point a2 is calculated as a predicted value in a prediction mode indicated by a prediction mode value of "2". In this way, the prediction mode value indicating a prediction mode that calculates attribute information of point b1 as a predicted value is smaller than the prediction mode value indicating a prediction mode that calculates attribute information of point a2, which is located farther away from point b2 than point b1, as a predicted value.

[0429] This allows a small prediction mode value to be assigned to a point that is close in distance and therefore likely to be selected, thereby reducing the number of bits required to encode the prediction mode value. Also, a small prediction mode value may be preferentially assigned to a 3D point that belongs to the same LoD as the 3D point to be encoded.

[0430] 70 is a diagram showing a third example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the third example is an example in which attribute information used for the predicted values ​​is a value based on color information (YUV) of surrounding three-dimensional points. In this way, the attribute information used for the predicted values ​​may be color information indicating the color of the three-dimensional points.

[0431] As shown in FIG. 70, predicted values ​​calculated in a prediction mode indicated by a prediction mode value of "0" are averages of the respective YUV components that define the YUV color space. Specifically, the predicted values ​​include a weighted average Yave of Yb1, Ya2, Ya1, and Ya0, which are Y component values ​​corresponding to points b1, a2, a1, and a0, respectively; a weighted average Uave of Ub1, Ua2, Ua1, and Ua0, which are U component values ​​corresponding to points b1, a2, a1, and a0, respectively; and a weighted average Vave of Vb1, Va2, Va1, and Va0, which are V component values ​​corresponding to points b1, a2, a1, and a0, respectively. Furthermore, predicted values ​​calculated in prediction modes indicated by prediction mode values ​​of "1" to "4" include color information of the surrounding three-dimensional points b1, a2, a1, and a0, respectively. The color information is represented by a combination of values ​​of the Y component, U component, and V component.

[0432] In Figure 70, the color information is shown using values ​​defined in the YUV color space, but it is not limited to the YUV color space and may be shown using values ​​defined in the RGB color space or in other color spaces.

[0433] In this way, in calculating the predicted value, two or more averages or attribute information may be calculated as the predicted value of the prediction mode. Furthermore, the two or more averages or attribute information may each indicate values ​​of two or more components that define the color space.

[0434] For example, when a prediction mode indicated by a prediction mode value of "2" in the table of Fig. 70 is selected, the Y component, U component, and V component of the attribute value of the 3D point to be encoded may be used as predicted values ​​Ya2, Ua2, and Va2, respectively, for encoding. In this case, the prediction mode value "2" is added to the bitstream.

[0435] 71 is a diagram showing a fourth example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the fourth example is an example in which attribute information used for the predicted values ​​is a value based on reflectance information of surrounding three-dimensional points. The reflectance information is, for example, information indicating reflectance R.

[0436] 71, the predicted value calculated in a prediction mode indicated by a prediction mode value of "0" is the weighted average Rave of the reflectances Rb1, Ra2, Ra1, and Ra0 corresponding to points b1, a2, a1, and a0, respectively. Also, the predicted values ​​calculated in prediction modes indicated by prediction mode values ​​of "1" to "4" are the reflectances Rb1, Ra2, Ra1, and Ra0 of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0437] For example, when a prediction mode indicated by a prediction mode value of "3" in the table of Fig. 71 is selected, the reflectance of the attribute value of the 3D point to be coded may be used as the predicted value Ra1 for coding. In this case, the prediction mode value "3" is added to the bitstream.

[0438] As shown in Figures 70 and 71, the attribute information may include first attribute information and second attribute information of a type different from the first attribute information. The first attribute information may be, for example, color information. The second attribute information may be, for example, reflectance information. In calculating the predicted value, the first predicted value may be calculated using the first attribute information, and the second predicted value may be calculated using the second attribute information.

[0439] When the attribute information has multiple components, such as color information in a YUV color space or an RGB color space, a predicted value may be calculated using a prediction mode for each component. For example, in the case of a YUV space, predicted values ​​using the Y component, the U component, and the V component may each be calculated using a prediction mode selected for the respective component. For example, a prediction mode value may be selected for each of a prediction mode Y for calculating a predicted value using the Y component, a prediction mode U for calculating a predicted value using the U component, and a prediction mode V for calculating a predicted value using the V component. In this case, the values ​​in the tables of Figures 72 to 74 described below are used as prediction mode values ​​indicating the prediction mode for each component, and these prediction mode values ​​may each be added to the bitstream. Note that although the YUV color space has been described above, the same can be applied to an RGB color space.

[0440] In addition, predicted values ​​including two or more components among the multiple components of the attribute information may be calculated using a common prediction mode.For example, in the case of a YUV color space, a prediction mode value may be selected for each of a prediction mode Y for calculating a predicted value using the Y component and a prediction mode UV for calculating a predicted value using the UV component.In this case, the prediction mode value indicating the prediction mode of each component may use the values ​​of the tables in Figures 72 and 75 described below, and these prediction mode values ​​may be added to the bitstream.

[0441] 72 is a diagram showing a fifth example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the fifth example is an example in which attribute information used for the predicted value is the Y component value of color information of surrounding 3D points.

[0442] 72, the predicted value calculated in a prediction mode Y indicated by a prediction mode value of "0" is a weighted average Yave of Y component values ​​Yb1, Ya2, Ya1, and Ya0 corresponding to points b1, a2, a1, and a0, respectively. Also, the predicted values ​​calculated in prediction modes indicated by prediction mode values ​​of "1" to "4" are the Y component values ​​Yb1, Ya2, Ya1, and Ya0 of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0443] For example, when a prediction mode Y indicated by a prediction mode value of "2" in the table of Fig. 72 is selected, the Y component of the attribute value of the 3D point to be coded may be used as a predicted value Ya2 for coding. In this case, the prediction mode value "2" is added to the bitstream.

[0444] 73 is a diagram showing a sixth example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the sixth example is an example in which attribute information used for the predicted value is the value of the U component of color information of surrounding 3D points.

[0445] 73, the predicted value calculated in a prediction mode U indicated by a prediction mode value of "0" is a weighted average Uave of U component values ​​Ub1, Ua2, Ua1, and Ua0 corresponding to points b1, a2, a1, and a0, respectively. Also, the predicted values ​​calculated in prediction modes indicated by prediction mode values ​​of "1" to "4" are U component values ​​Ub1, Ua2, Ua1, and Ua0 of surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0446] For example, when a prediction mode U indicated by a prediction mode value of "1" in the table of Fig. 73 is selected, the U component of the attribute value of the 3D point to be coded may be used as a predicted value Ub1 for coding. In this case, the prediction mode value "1" is added to the bitstream.

[0447] 74 is a diagram showing a seventh example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the seventh example is an example in which attribute information used for the predicted value is the value of the V component of color information of surrounding 3D points.

[0448] 74, the predicted value calculated in a prediction mode V indicated by a prediction mode value of "0" is the weighted average Vave of V component values ​​Vb1, Va2, Va1, and Va0 corresponding to points b1, a2, a1, and a0, respectively. Also, the predicted values ​​calculated in prediction modes indicated by prediction mode values ​​of "1" to "4" are the V component values ​​Vb1, Va2, Va1, and Va0 of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0449] For example, when a prediction mode V indicated by a prediction mode value of "4" in the table of Fig. 74 is selected, the V component of the attribute value of the 3D point to be coded may be used as a predicted value Va0 for coding. In this case, the prediction mode value "4" is added to the bitstream.

[0450] 75 is a diagram showing an eighth example of a table indicating predicted values ​​calculated in each prediction mode according to Embodiment 5. Specifically, the eighth example is an example in which attribute information used for the predicted values ​​is the U component values ​​and V component values ​​of color information of surrounding 3D points.

[0451] 75, predicted values ​​calculated in a prediction mode U indicated by a prediction mode value of "0" include a weighted average Uave of U component values ​​Ub1, Ub2, Ua1, and Ua0 corresponding to points b1, a2, a1, and a0, respectively, and a weighted average Vave of V component values ​​Vb1, Vb2, Va1, and Va0 corresponding to points b1, a2, a1, and a0, respectively. Also, predicted values ​​calculated in prediction modes indicated by prediction mode values ​​of "1" to "4" include U component values ​​and V component values ​​of surrounding three-dimensional points b1, a2, a1, and a0, respectively.

[0452] For example, when a prediction mode UV indicated by a prediction mode value of "1" in the table of Fig. 75 is selected, the U component and V component of the attribute value of the 3D point to be encoded may be used as predicted values ​​Ub1 and Vb1, respectively, for encoding. In this case, the prediction mode value "1" is added to the bitstream.

[0453] The prediction mode during encoding may be selected by RD optimization. For example, it is considered that the cost (P) when a certain prediction mode P is selected is calculated, and the prediction mode P that minimizes the cost (P) is selected. The cost (P) may be calculated by formula D1 using the prediction residual (P) when the predicted value of the prediction mode P is used, the number of bits (P) required to encode the prediction mode P, and the adjustment parameter λ value.

[0454] cost(P)=abs(residual(P))+λ×bit(P)...(Formula D1)

[0455] abs(x) indicates the absolute value of x. You can also use the square of x instead of abs(x).

[0456] By using the above formula D1, it is possible to select a prediction mode that takes into consideration the balance between the magnitude of the prediction residual and the number of bits required to encode the prediction mode. Note that the adjustment parameter λ may be set to a different value depending on the value of the quantization scale. For example, when the quantization scale is small (at a high bit rate), the λ value may be reduced to select a prediction mode that reduces the prediction residual (P) and improve prediction accuracy as much as possible, and when the quantization scale is large (at a low bit rate), the λ value may be increased to select an appropriate prediction mode while taking into consideration the number of bits (P) required to encode the prediction mode P.

[0457] A small quantization scale is, for example, smaller than the first quantization scale. A large quantization scale is, for example, larger than a second quantization scale that is equal to or larger than the first quantization scale. The smaller the quantization scale, the smaller the λ value may be set.

[0458] The prediction residual residual(P) is calculated by subtracting the predicted value of prediction mode P from the attribute value of the 3D point to be coded. Note that instead of using the prediction residual residual(P) when calculating the cost, the prediction residual residual(P) may be quantized and dequantized, and added to the predicted value to obtain a decoded value, and the difference (encoding error) between the attribute value of the original 3D point and the decoded value when prediction mode P is used may be reflected in the cost value. This makes it possible to select a prediction mode with a small coding error.

[0459] The number of bits (P) required to encode a prediction mode P may be, for example, the number of bits after binarization when the prediction mode is binarized and encoded. For example, when the number of prediction modes M=5, as shown in FIG. 76, a prediction mode value indicating the prediction mode may be binarized using a truncated unary code with a maximum value of 5 using the number of prediction modes M. In this case, 1 bit is used as the number of bits (P) required to encode each prediction mode value when the prediction mode value is "0", 2 bits when the prediction mode value is "1", 3 bits when the prediction mode value is "2", and 4 bits when the prediction mode value is "4" or "5". By using a truncated unary code, the number of bits becomes smaller as the prediction mode value becomes smaller. Therefore, it is possible to reduce the amount of code of a prediction mode value indicating a prediction mode that calculates a predicted value that is likely to be selected, for example, that is likely to have the smallest cost (P), such as an average value calculated as a predicted value when the prediction mode value is "0" or attribute information of a 3D point calculated as a predicted value when the prediction mode value is "1", that is, attribute information of a 3D point close to the 3D point to be encoded.

[0460] Furthermore, when the maximum value of the number of prediction modes has not been determined, the prediction mode value indicating the prediction mode may be binarized using unary code as shown in Figure 77. Furthermore, when the occurrence probability of each prediction mode is close, the prediction mode value indicating the prediction mode may be binarized using fixed code as shown in Figure 78 to reduce the amount of code.

[0461] Note that the number of bits (P) required to encode a prediction mode value indicating a prediction mode P may be determined by arithmetically encoding the binarized data of the prediction mode value indicating the prediction mode P, and the code amount after arithmetic encoding may be set as the value of bit(P). This allows the cost to be calculated using a more accurate number of required bits (P), making it possible to select a more appropriate prediction mode.

[0462] 76 is a diagram showing a first example of a binarization table when binarizing and encoding prediction mode values ​​according to Embodiment 5. Specifically, the first example is an example in which prediction mode values ​​are binarized using truncated unary code when the number of prediction modes M=5.

[0463] 77 is a diagram showing a second example of a binarization table when binarizing and encoding prediction mode values ​​according to Embodiment 5. Specifically, the second example is an example in which prediction mode values ​​are binarized using unary code when the number of prediction modes M=5.

[0464] 78 is a diagram showing a third example of a binarization table when binarizing and encoding prediction mode values ​​according to Embodiment 5. Specifically, the third example is an example in which prediction mode values ​​are binarized using fixed code when the number of prediction modes M=5.

[0465] The prediction mode value indicating the prediction mode (PredMode) may be binarized and then arithmetically coded, and added to the bitstream.As described above, the prediction mode value may be binarized by a truncated unary code using the value of the number of prediction modes M, for example.In this case, the maximum number of bits of the prediction mode value after binarization is M-1.

[0466] Furthermore, the binary data after binarization may be arithmetically coded using a coding table. In this case, for example, coding efficiency may be improved by switching coding tables for each bit of the binary data. Furthermore, in order to reduce the number of coding tables, the first bit of the binary data may be coded using coding table A for one bit, and each bit of the remaining bits may be coded using coding table B for the remaining bits. For example, when coding the binarized data "1110" with a prediction mode value of "3" shown in FIG. 79, the first bit "1" may be coded using coding table A, and each bit of the remaining bits "110" may be coded using coding table B.

[0467] 79 is a diagram illustrating an example of encoding binary data of a binarization table when binarizing and encoding prediction modes according to Embodiment 5. The binarization table in FIG. 79 is an example of binarizing prediction mode values ​​using truncated unary code when the number of prediction modes M=5.

[0468] This makes it possible to improve coding efficiency by switching coding tables according to the bit position of binary data while suppressing the number of coding tables. Note that when coding the remaining bits, it is also possible to switch coding tables for each bit and perform arithmetic coding, or to switch coding tables according to the arithmetic coding results and perform decoding.

[0469] When binarizing and encoding prediction mode values ​​using a truncated unary code using the number of prediction modes M, the number of prediction modes M used in the truncated unary code may be added to a header or the like of a bitstream so that the prediction mode can be identified from the binary data decoded on the decoding side. Also, a possible value MaxM of the number of prediction modes may be specified by a standard or the like, and a value of MaxM-M (M<=MaxM) may be added to the header. Also, the number of prediction modes M may be specified in a profile or level of a standard or the like without being added to the stream.

[0470] It is possible to arithmetically code a prediction mode value binarized using a truncated unary code by switching coding tables between the one-bit part and the remaining part as described above. The occurrence probabilities of 0 and 1 in each coding table may be updated according to the value of the binary data that actually occurs. Also, the occurrence probabilities of 0 and 1 in either coding table may be fixed. This may reduce the number of updates to the occurrence probabilities and reduce the amount of processing. For example, the occurrence probability of the one-bit part may be updated, and the occurrence probability of the remaining bit part may be fixed.

[0471] 80 is a flowchart showing an example of encoding of prediction mode values ​​according to Embodiment 5. FIG. 81 is a flowchart showing an example of decoding of prediction mode values ​​according to Embodiment 5.

[0472] As shown in FIG. 80, in encoding a prediction mode value, first, the prediction mode value is binarized by a truncated unary code using the number of prediction modes M (S3401).

[0473] Next, the binary data of the truncated unary code is arithmetically coded (S3402), so that the binary data is included as a prediction mode in the bitstream.

[0474] Furthermore, as shown in FIG. 81, in decoding the prediction mode value, first, the bit stream is arithmetically decoded using the number of prediction modes M to generate binary data of a truncated unary code (S3403).

[0475] Next, a prediction mode value is calculated from the binary data of the truncated unary code (S3404).

[0476] Although an example of binarizing a prediction mode value indicating a prediction mode (PredMode) using a truncated unary code using the value of the prediction mode number M has been shown, this is not necessarily limited thereto. For example, the prediction mode value may be binarized using a truncated unary code using the number L (L<=M) of predicted values ​​assigned to the prediction mode. For example, when the prediction mode number M=5, if there are two surrounding three-dimensional points available for predicting a three-dimensional point to be coded, there may be cases where three prediction modes are available, as shown in Figure 82, and the remaining two prediction modes are not available. For example, when the prediction mode number M=5 as shown in Figure 82, there may be cases where there are two surrounding three-dimensional points available for prediction, as shown in Figure 82, and no predicted values ​​are assigned to the prediction modes with prediction mode values ​​of "3" and "4".

[0477] In this case, as shown in Fig. 83 , by binarizing the prediction mode value with truncated unary code using the value L assigned to the prediction mode as the maximum value, it is possible to reduce the number of bits after binarization compared to when truncated unary code is used with the number of prediction modes M. For example, in this case, since L = 3, the number of bits can be reduced by binarizing with truncated unary code using the maximum value 3. In this way, the number of bits after binarization of the prediction mode value may be reduced by binarizing with truncated unary code using the number L assigned to the prediction mode as the maximum value.

[0478] The binary data after binarization may be arithmetically coded using a coding table. In this case, coding efficiency may be improved by, for example, switching coding tables for each bit of the binary data. Furthermore, to reduce the number of coding tables, the first bit of the binarized data may be coded using coding table A for one bit, and each bit of the remaining bits may be coded using coding table B for the remaining bits. For example, when coding the binarized data "11" with a prediction mode value of "2" shown in FIG. 83, the first bit "1" may be coded using coding table A, and the remaining bits "1" may be coded using coding table B.

[0479] 83 is a diagram illustrating an example of encoding binary data of a binarization table when binarizing and encoding a prediction mode according to Embodiment 5. The binarization table in Fig. 83 is an example of binarizing prediction mode values ​​with a truncated unary code when the number L of predicted values ​​assigned to prediction modes is 3.

[0480] This makes it possible to improve coding efficiency by switching coding tables according to the bit position of binary data while suppressing the number of coding tables. Note that when coding the remaining bits, it is also possible to switch coding tables for each bit and perform arithmetic coding, or to switch coding tables according to the arithmetic coding results and perform decoding.

[0481] When the prediction mode value is binarized and encoded using a truncated unary code using a number L to which a prediction value is assigned, the number L may be calculated by assigning a prediction value to the prediction mode in the same manner as during encoding, and the calculated L may be used to decode the prediction mode so that the prediction mode can be identified from the decoded binary data on the decoding side.

[0482] It is possible to arithmetically code a prediction mode value binarized using a truncated unary code by switching coding tables between the one-bit part and the remaining part as described above. The occurrence probabilities of 0 and 1 in each coding table may be updated according to the value of the binary data that actually occurs. Also, the occurrence probabilities of 0 and 1 in either coding table may be fixed. This may reduce the number of updates to the occurrence probabilities and reduce the amount of processing. For example, the occurrence probability of the one-bit part may be updated, and the occurrence probability of the remaining bit part may be fixed.

[0483] 84 is a flowchart showing another example of encoding of prediction mode values ​​according to Embodiment 5. FIG. 85 is a flowchart showing another example of decoding of prediction mode values ​​according to Embodiment 5.

[0484] As shown in FIG. 84, in encoding prediction mode values, first, the number L of prediction values ​​assigned to prediction modes is calculated (S3411).

[0485] Next, the prediction mode value is binarized by a truncated unary code using the number L (S3412).

[0486] Next, the binary data of the truncated unary code is arithmetically coded (S3413).

[0487] As shown in FIG. 85, in decoding the prediction mode value, first, the number L of prediction values ​​assigned to the prediction modes is calculated (S3414).

[0488] Next, the bit stream is arithmetically decoded using the number L to generate binary data of a truncated unary code (S3415).

[0489] Next, a prediction mode value is calculated from the binary data of the truncated unary code (S3416).

[0490] The prediction mode value does not have to be added to every attribute value. For example, if a certain condition is satisfied, the prediction mode may be fixed and the prediction mode value may not be added to the bitstream, and if a certain condition is not satisfied, a prediction mode may be selected and the prediction mode value may be added to the bitstream. For example, if condition A is satisfied, the prediction mode value may be fixed to "0" and a predicted value may be calculated from the average value of surrounding three-dimensional points, and if condition A is not satisfied, one prediction mode may be selected from multiple prediction modes and a prediction mode value indicating the selected prediction mode may be added to the bitstream.

[0491] One example of condition A is that the maximum absolute difference value maxdiff of the attribute values ​​(a[0] to a[N-1]) of N surrounding three-dimensional points (already coded and decoded) around the three-dimensional point to be coded is calculated, and the maximum absolute difference value maxdiff is less than a threshold value Thfix. If the maximum absolute difference value of the attribute values ​​of the surrounding three-dimensional points is less than the threshold value Thfix, it is determined that there is little difference in the attribute values ​​of each three-dimensional point, and that there will be no difference in the predicted values ​​even if a prediction mode is selected. By fixing the prediction mode value to "0" and not coding the prediction mode value, it is possible to generate an appropriate predicted value while reducing the amount of code required to code the prediction mode.

[0492] The threshold value Thfix may be added to the header of the bitstream, or the like, and the encoder may change the value of the threshold value Thfix for encoding. For example, when encoding at a high bitrate, the encoder may add a smaller value of the threshold value Thfix to the header than when encoding at a low bitrate, thereby increasing the number of cases in which a prediction mode is selected and encoded, thereby encoding so as to reduce the prediction residual as much as possible. Furthermore, when encoding at a low bitrate, the encoder may add a larger value of the threshold value Thfix to the header than when encoding at a high bitrate, thereby encoding with a fixed prediction mode. In this way, by increasing the number of cases in which a prediction mode is fixed and encoded at a low bitrate, it is possible to improve encoding efficiency while reducing the amount of bits for encoding the prediction mode. Furthermore, the threshold value Thfix may not be added to the bitstream, but may be specified in the profile or level of the standard.

[0493] The N neighboring 3D points of the 3D point to be coded that are used for prediction are the N coded and decoded 3D points whose distance from the 3D point to be coded is less than a threshold value THd. The maximum value of N may be added to the bitstream as NumNeighborPoint. The value of N does not always have to match the value of NumNeighborPoint, for example, when the number of neighboring coded and decoded 3D points is less than the value of NumNeighborPoint.

[0494] Although an example has been shown in which the prediction mode value is fixed to "0" if the maximum absolute difference value maxdiff of the attribute values ​​of the 3D points surrounding the 3D point to be coded used for prediction is smaller than the threshold value Thfix[i], the present invention is not limited to this, and the prediction mode value may be fixed to any one of "0" to "M-1". Furthermore, the fixed prediction mode value may be added to the bitstream.

[0495] Figure 86 is a flowchart showing an example of a process for determining whether to fix a prediction mode value in accordance with condition A during encoding according to Embodiment 5. Figure 87 is a flowchart showing an example of a process for determining whether to fix a prediction mode value in accordance with condition A during decoding according to Embodiment 5 or to decode.

[0496] As shown in FIG. 86, first, the three-dimensional data encoding device calculates the maximum absolute difference value maxdiff of the attribute values ​​at N three-dimensional points surrounding the three-dimensional point to be encoded (S3421).

[0497] Next, the three-dimensional data encoding device determines whether the maximum absolute difference value maxdiff is less than a threshold value Thfix (S3422). Note that the threshold value Thfix may be encoded and added to the header of the stream, etc.

[0498] If the maximum absolute difference value maxdiff is less than the threshold value Thfix (Yes in S3422), the three-dimensional data encoding device determines the prediction mode value to be "0" (S3423).

[0499] On the other hand, if the maximum absolute difference value maxdiff is equal to or greater than the threshold value Thfix (No in S3422), the three-dimensional data encoding device selects one prediction mode from the plurality of prediction modes (S3424). Details of the prediction mode selection process will be described later with reference to FIG. 94.

[0500] Then, the three-dimensional data encoding device arithmetically encodes the prediction mode value indicating the selected prediction mode (S3425). Specifically, the three-dimensional data encoding device arithmetically encodes the prediction mode value by executing steps S3401 and S3402 described in Fig. 80. Note that the three-dimensional data encoding device may arithmetically encode the prediction mode PredMode by binarizing it with a truncated unary code using the number of prediction modes to which the prediction value is assigned. In other words, the three-dimensional data encoding device may arithmetically encode the prediction mode value by executing steps S3411 to S3413 described in Fig. 84.

[0501] The three-dimensional data encoding device calculates a predicted value for the prediction mode determined in step S3423 or the prediction mode selected in step S3425, and outputs the calculated predicted value (S3426). When using the prediction mode value determined in step S3423, the three-dimensional data encoding device calculates the average of the attribute values ​​of the surrounding N three-dimensional points as the predicted value for the prediction mode indicated by the prediction mode value "0".

[0502] Also, as shown in Fig. 87, the three-dimensional data decoding device first calculates the maximum absolute difference value maxdiff of the attribute values ​​of N three-dimensional points surrounding the three-dimensional point to be decoded (S3431). Note that the maximum absolute difference value maxdiff may also be calculated as shown in Fig. 88. The maximum absolute difference value maxdiff is the maximum value of the calculated absolute values ​​of the differences between all possible pairs when two of the N surrounding three-dimensional points are selected as a pair, for example.

[0503] Next, the three-dimensional data decoding device determines whether or not the maximum absolute difference value maxdiff is less than a threshold value Thfix (S3432). Note that the threshold value Thfix may be set by decoding the header of the stream, etc.

[0504] If the maximum absolute difference value maxdiff is less than the threshold value Thfix (Yes in S3432), the three-dimensional data decoding device determines the prediction mode value to be "0" (S3433).

[0505] On the other hand, if the maximum absolute difference value maxdiff is equal to or greater than the threshold value Thfix (No in S3432), the three-dimensional data decoding device decodes the prediction mode value from the bitstream (S3434).

[0506] The three-dimensional data decoding device calculates a predicted value of the prediction mode indicated by the prediction mode value determined in step S3433 or the prediction mode value decoded in step S3434, and outputs the calculated predicted value (S3435). When using the prediction mode value determined in step S3433, the three-dimensional data decoding device calculates the average of the attribute values ​​of the surrounding N three-dimensional points as the predicted value of the prediction mode indicated by the prediction mode value "0".

[0507] Fig. 89 is a diagram illustrating an example of syntax according to Embodiment 5. NumLod, NumNeighborPoint[i], NumPredMode[i], Thfix[i], and NumOfPoint[i] in the syntax of Fig. 89 will be described in order.

[0508] NumLoD indicates the number of levels of LoD.

[0509] NumNeighborPoint[i] indicates the upper limit value of the number of surrounding points used for generating the predicted value of the three-dimensional points belonging to level i. When the number of surrounding points M is less than NumNeighborPoint[i] (M < NumNeighborPoint[i]), the predicted value may be calculated using M surrounding points. Also, when it is not necessary to divide the value of NumNeighborPoint[i] for each LoD, one NumNeighborPoint may be added to the header.

[0510] NumPredMode[i] indicates the total number (M) of prediction modes used for predicting the attribute value of level i. Note that the possible value MaxM of the number of prediction modes is defined by a standard or the like, and the value (0 < M <= MaxM) of MaxM - M may be added to the header as NumPredMode[i] and encoded by binarizing it with a truncated unary code of the maximum value MaxM - 1. Also, the number of prediction modes NumPredMode[i] may be defined by a profile or level of a standard or the like without being added to the stream. Also, the number of prediction modes may be defined by NumNeighborPoint[i] + NumPredMode[i]. Also, when it is not necessary to divide the value of NumPredMode[i] for each LoD, one NumPredMode may be added to the header.

[0511] Thfix[i] indicates the threshold value of the maximum absolute difference value for determining whether to fix the prediction mode of level i. If the maximum absolute difference value of the attribute values of the surrounding three-dimensional points used for prediction is less than Thfix[i], the prediction mode is fixed to 0. Note that Thfix[i] may be defined by a profile or level of a standard or the like without being added to the stream. Also, when it is not necessary to divide the value of Thfix[i] for each LoD, one Thfix may be added to the header.

[0512] NumOfPoint[i] indicates the number of 3D points belonging to layer i. Note that if the total number of 3D points, AllNumOfPoint, is added to a separate header, NumOfPoint[NumLoD-1] (the number of 3D points belonging to the lowest layer) is not added to the header.

number

[0513] As an example of setting NumPredMode[i], the value of NumPredMode[i] may be set larger for higher layers where the distance between 3D points belonging to the LoD is greater and therefore the prediction accuracy is lower, thereby increasing the number of selectable prediction modes. Alternatively, the value of NumPredMode[i] may be set smaller for lower layers where the prediction accuracy is higher, thereby reducing the amount of bits required for encoding the prediction modes. These settings increase the number of selectable prediction modes in higher layers, reducing the prediction residual, and reduce the amount of coding for the prediction modes in lower layers, thereby improving encoding efficiency.

[0514] In addition, as an example of setting Thfix[i], the higher the layer where the distance between the three-dimensional points belonging to the LoD is farther and the more difficult the prediction is to be, the smaller the value of Thfix[i] may be set, thereby increasing the number of cases where a prediction mode is selected.Also, the lower the layer where the prediction is more likely to be accurate, the larger the value of Thfix[i] may be set, thereby fixing the prediction mode and reducing the amount of bits required for encoding the prediction mode.With these settings, the number of cases where a prediction mode is selected in the higher layer is increased, thereby reducing the prediction residual, and the prediction mode in the lower layer is fixed, thereby reducing the amount of code for the prediction mode, thereby improving encoding efficiency.

[0515] The above NumLod, NumNeighborPoint[i], NumPredMode[i], Thfix[i], or NumOfPoint[i] may be entropy-coded and added to the header. For example, each value may be binarized and arithmetically coded. Also, each value may be coded at a fixed length to reduce the amount of processing.

[0516] Fig. 90 is a diagram showing an example of syntax according to Embodiment 5. PredMode, n-bit code, and remaining code in the syntax of Fig. 90 will be described in order.

[0517] PredMode indicates a prediction mode for encoding and decoding the attribute value of the j-th 3D point in layer i. PredMode takes on values ​​from "0" to "M-1" (M is the total number of prediction modes). If PredMode is not present in the bitstream (if the condition maxdiff >= Thfix[i] && NumPredMode[i] > 1 is not satisfied), PredMode may be estimated to be 0. Note that PredMode is not limited to "0" and may take any value from "0" to "M-1" as an estimated value. Furthermore, if PredMode is not present in the bitstream, the estimated value may be added to a separate header or the like. Furthermore, PredMode may be binarized with a truncated unary code using the number of prediction modes to which the predicted value is assigned, and then arithmetically coded.

[0518] The n-bit code indicates coded data of the prediction residual of the attribute information value. The bit length of the n-bit code depends on the value of R_TH[i]. For example, if the value indicated by R_TH[i] is 63, the bit length of the n-bit code is 6 bits, and if the value indicated by R_TH[i] is 255, the bit length is 8 bits.

[0519] The remaining code indicates the coded data of the prediction residual of the attribute information value that has been coded using Exponential-Golomb coding. The remaining code is decoded when the n-bit code is the same as R_TH[i], and the value of the n-bit code and the value of the remaining code are added together to decode the prediction residual. Note that if the n-bit code is not the same value as R_TH[i], the remaining code does not need to be decoded.

[0520] The flow of processing in the three-dimensional data encoding device will be described below. Fig. 91 is a flowchart of three-dimensional data encoding processing by the three-dimensional data encoding device according to the fifth embodiment.

[0521] First, the three-dimensional data encoding device encodes position information (geometry) (S3441). For example, the three-dimensional data is encoded using an octree representation.

[0522] If the position of a three-dimensional point changes due to quantization or the like after encoding the position information, the three-dimensional data encoding device reallocates the attribute information of the original three-dimensional point to the changed three-dimensional point (S3442). For example, the three-dimensional data encoding device performs the reallocation by interpolating the value of the attribute information according to the amount of change in position. For example, the three-dimensional data encoding device detects N pre-change three-dimensional points that are close to the changed three-dimensional position, and calculates a weighted average of the attribute information values ​​of the N three-dimensional points. For example, the three-dimensional data encoding device determines a weight for the weighted average based on the distance from the changed three-dimensional position to each of the N three-dimensional points. The three-dimensional data encoding device then determines the value obtained by the weighted average as the value of the attribute information of the changed three-dimensional point. Furthermore, if two or more three-dimensional points change to the same three-dimensional position due to quantization or the like, the three-dimensional data encoding device may assign the average value of the attribute information of the two or more pre-change three-dimensional points as the value of the attribute information of the changed three-dimensional point.

[0523] Next, the three-dimensional data encoding device encodes the reallocated attribute information (Attribute) (S3443). For example, when encoding multiple types of attribute information, the three-dimensional data encoding device may encode the multiple types of attribute information in order. For example, when encoding color and reflectance as attribute information, the three-dimensional data encoding device may generate a bitstream in which the encoding result of reflectance is added after the encoding result of color. Note that the order in which the encoding results of the multiple attribute information are added to the bitstream is not limited to this order and may be any order.

[0524] The three-dimensional data encoding device may also add information indicating the start location of the encoded data for each piece of attribute information in the bitstream to a header or the like. This allows the three-dimensional data decoding device to selectively decode attribute information that needs to be decoded, thereby omitting the decoding process for attribute information that does not need to be decoded. This reduces the amount of processing required by the three-dimensional data decoding device. The three-dimensional data encoding device may also encode multiple types of attribute information in parallel and combine the encoding results into a single bitstream. This allows the three-dimensional data encoding device to encode multiple types of attribute information at high speed.

[0525] 92 is a flowchart of the attribute information encoding process (S3443) according to Embodiment 5. First, the three-dimensional data encoding device sets the LoD (S3451). That is, the three-dimensional data encoding device assigns each three-dimensional point to one of a plurality of LoDs.

[0526] Next, the three-dimensional data encoding device starts a loop for each LoD (S3452). That is, the three-dimensional data encoding device repeatedly performs the processes of steps S3453 to S3461 for each LoD.

[0527] Next, the three-dimensional data encoding device starts a loop for each three-dimensional point (S3453). That is, the three-dimensional data encoding device repeatedly performs the processes of steps S3454 to S3460 for each three-dimensional point.

[0528] First, the three-dimensional data encoding device searches for a plurality of surrounding points, which are three-dimensional points existing around the target three-dimensional point to be processed, and which are used to calculate a predicted value of the target three-dimensional point (S3454).

[0529] Next, the three-dimensional data encoding device calculates a predicted value P of the target three-dimensional point (S3455). A specific example of the process of calculating the predicted value P will be described later with reference to FIG.

[0530] Next, the 3D data encoding device calculates a prediction residual, which is the difference between the attribute information of the target 3D point and the predicted value (S3456).

[0531] Next, the three-dimensional data encoding device calculates a quantized value by quantizing the prediction residual (S3457).

[0532] Next, the three-dimensional data encoding device arithmetically encodes the quantized value (S3458).

[0533] Furthermore, the three-dimensional data encoding device calculates an inverse quantized value by inverse quantizing the quantized value (S3459).

[0534] Next, the three-dimensional data encoding device generates a decoded value by adding the predicted value to the inverse quantized value (S3460).

[0535] Next, the three-dimensional data encoding device ends the three-dimensional point-by-point loop (S3461).

[0536] The three-dimensional data encoding device also ends the loop for each LoD (S3462).

[0537] FIG. 93 is a flowchart of the process of calculating a predicted value (S3455) in the three-dimensional data encoding device according to the fifth embodiment.

[0538] First, the three-dimensional data encoding device calculates a weighted average of the attribute values ​​of N three-dimensional points surrounding the target three-dimensional point that can be used to predict the predicted value of the target three-dimensional point being processed, and assigns the calculated weighted average to a prediction mode with a prediction mode value of "0" (S3420).

[0539] Next, the three-dimensional data encoding device executes steps S3421 to S3426 described with reference to FIG. 86, thereby outputting the predicted value of the target three-dimensional point.

[0540] FIG. 94 is a flowchart of the prediction mode selection process (S3424) according to the fifth embodiment.

[0541] First, the three-dimensional data encoding device assigns values ​​incremented by one from "1" to "N" as prediction mode values ​​to the attribute information of N three-dimensional points surrounding the target three-dimensional point, in order from closest to the target three-dimensional point (S3427). Note that since prediction mode values ​​are assigned from "0" to "N", a total of N+1 prediction modes are generated. If N+1 exceeds the maximum number of prediction modes M (NumPredMode) that can be added to the bitstream, the three-dimensional data encoding device may generate up to M prediction modes.

[0542] Next, the three-dimensional data encoding apparatus calculates the cost of each prediction mode and selects the prediction mode with the smallest cost (S3428).

[0543] FIG. 95 is a flowchart showing the process (S3428) of selecting a prediction mode with the minimum cost according to the fifth embodiment.

[0544] First, the three-dimensional data encoding device sets initial values ​​to i=0 and mincost=∞ (S3471). The set initial values ​​of i and mincost are stored in the memory of the three-dimensional data encoding device.

[0545] Next, the three-dimensional data encoding device calculates the cost cost[i] of the i-th prediction mode PredMode[i] using, for example, equation D1 (S3472).

[0546] Next, the three-dimensional data encoding device determines whether the calculated cost cost[i] is smaller than mincost stored in the memory (S3473).

[0547] Next, if the calculated cost cost[i] is smaller than the mincost stored in memory (Yes in S3473), the three-dimensional data encoding device sets mincost = cost[i], sets the prediction mode to predmode[i] (S3474), and proceeds to step S3475. That is, the value of mincost stored in memory is updated to the value of cost[i], and predmode[i] is stored in memory as the prediction mode.

[0548] On the other hand, if the calculated cost cost[i] is equal to or greater than mincost stored in the memory (No in S3473), the three-dimensional data encoding device proceeds to step S3475.

[0549] Next, the three-dimensional data encoding device increments the value of i by 1 (S3475).

[0550] Next, the three-dimensional data encoding apparatus determines whether i is smaller than the number of prediction modes (S3476).

[0551] If i is smaller than the number of prediction modes (Yes in S3476), the three-dimensional data encoding device returns to step S3472; if not (No in S3476), the three-dimensional data encoding device ends the process of selecting the prediction mode with the smallest cost.

[0552] The flow of processing in the three-dimensional data decoding device will be described below. Figure 96 is a flowchart of three-dimensional data decoding processing by the three-dimensional data decoding device according to embodiment 5. First, the three-dimensional data decoding device decodes position information (geometry) from the bitstream (S3444). For example, the three-dimensional data decoding device performs decoding using an octree representation.

[0553] Next, the three-dimensional data decoding device decodes attribute information (Attribute) from the bitstream (S3445). For example, when decoding multiple types of attribute information, the three-dimensional data decoding device may decode the multiple types of attribute information in order. For example, when decoding color and reflectance as attribute information, the three-dimensional data decoding device decodes the encoding result of color and the encoding result of reflectance in the order in which they are added to the bitstream. For example, when the encoding result of reflectance is added after the encoding result of color in the bitstream, the three-dimensional data decoding device decodes the encoding result of color, and then decodes the encoding result of reflectance. Note that the three-dimensional data decoding device may decode the encoding results of attribute information added to the bitstream in any order.

[0554] Furthermore, the three-dimensional data decoding device may obtain information indicating the start location of the encoded data for each piece of attribute information in the bitstream by decoding a header or the like. This allows the three-dimensional data decoding device to selectively decode attribute information that requires decoding, thereby omitting the decoding process for attribute information that does not require decoding. This reduces the amount of processing required by the three-dimensional data decoding device. Furthermore, the three-dimensional data decoding device may decode multiple types of attribute information in parallel and integrate the decoding results into a single three-dimensional point cloud. This allows the three-dimensional data decoding device to decode multiple types of attribute information at high speed.

[0555] 97 is a flowchart of the attribute information decoding process (S3445) according to the fifth embodiment. First, the three-dimensional data decoding device sets the LoD (S3481). That is, the three-dimensional data decoding device assigns each of the multiple three-dimensional points having decoded position information to one of the multiple LoDs. For example, this assignment method is the same as the assignment method used in the three-dimensional data encoding device.

[0556] Next, the three-dimensional data decoding device starts a loop for each LoD (S3482). That is, the three-dimensional data decoding device repeatedly performs the processes of steps S3483 to S3489 for each LoD.

[0557] Next, the three-dimensional data decoding device starts a loop for each three-dimensional point (S3483). That is, the three-dimensional data decoding device repeatedly performs the processes of steps S3484 to S3488 for each three-dimensional point.

[0558] First, the 3D data decoding device searches for multiple surrounding points, which are 3D points that exist around the target 3D point and are used to calculate a predicted value of the target 3D point (S3484). Note that this process is similar to the process in the 3D data encoding device.

[0559] Next, the three-dimensional data decoding device calculates a predicted value P of the target three-dimensional point (S3485).

[0560] Next, the three-dimensional data decoding device arithmetically decodes the quantized values ​​from the bitstream (S3486).

[0561] Furthermore, the three-dimensional data decoding device calculates inverse quantized values ​​by inverse quantizing the decoded quantized values ​​(S3487).

[0562] Next, the three-dimensional data decoding device generates a decoded value by adding the predicted value to the inverse quantized value (S3488).

[0563] Next, the three-dimensional data decoding device ends the three-dimensional point-by-point loop (S3489).

[0564] The three-dimensional data decoding device also ends the loop for each LoD (S3490).

[0565] FIG. 98 is a flowchart of the process of calculating the predicted value (S3485) in the three-dimensional data decoding device according to the fifth embodiment.

[0566] First, the three-dimensional data decoding device calculates a weighted average of the attribute values ​​of N three-dimensional points surrounding the target three-dimensional point that can be used to predict the predicted value of the target three-dimensional point being processed, and assigns the calculated weighted average to a prediction mode with a prediction mode value of "0" (S3430).

[0567] Next, the three-dimensional data decoding device executes steps S3431 to S3435 described in FIG. 87, thereby outputting the predicted value of the target three-dimensional point.

[0568] Note that instead of executing step S3430, after the determination in step S3432 is Yes, or when the prediction mode value decoded in step S3434 is "0," a weighted average of attribute values ​​of N 3D points around the target 3D point that can be used to predict the predicted value of the target 3D point to be processed may be calculated as a predicted value. This eliminates the need to calculate the average value in prediction modes other than those indicated by a prediction mode value of "0," making it possible to reduce the amount of processing.

[0569] FIG. 99 is a flowchart of the prediction mode decoding process (S3434) according to the fifth embodiment.

[0570] First, the 3D decoding device assigns values ​​from "1" to "N" in increments of 1 as prediction mode values ​​to the attribute information of N 3D points surrounding the target 3D point, in order from closest to the target 3D point (S3491). Note that since prediction mode values ​​are assigned from "0" to "N", a total of N+1 prediction modes are generated. If N+1 exceeds the maximum number of prediction modes M (NumPredMode) that can be added to the bitstream, the 3D data decoding device may generate up to M prediction modes.

[0571] Next, the three-dimensional data decoding device arithmetically decodes the prediction mode using the number of prediction modes (S3492). Specifically, the three-dimensional data decoding device may arithmetically decode the prediction mode by executing steps S3403 and S3404 described in Fig. 81. Furthermore, the three-dimensional data decoding device may arithmetically decode the prediction mode by executing steps S3414 to S3416 described in Fig. 85.

[0572] Fig. 100 is a block diagram showing the configuration of the attribute information encoding unit 3400 included in the three-dimensional data encoding device in embodiment 5. Fig. 100 shows details of the attribute information encoding unit out of the position information encoding unit, attribute information reallocation unit, and attribute information encoding unit included in the three-dimensional data encoding device.

[0573] The attribute information encoding unit 3400 includes an LoD generation unit 3401, a surrounding search unit 3402, a prediction unit 3403, a prediction residual calculation unit 3404, a quantization unit 3405, an arithmetic encoding unit 3406, an inverse quantization unit 3407, a decoded value generation unit 3408, and a memory 3409.

[0574] The LoD generation unit 3401 generates LoD using position information (geometry) of three-dimensional points.

[0575] The surrounding search unit 3402 searches for nearby 3D points adjacent to each 3D point using the LoD generation result by the LoD generation unit 3401 and distance information indicating the distance between each 3D point.

[0576] The prediction unit 3403 generates a predicted value of attribute information of a target 3D point to be coded. Specifically, the prediction unit 3403 assigns the predicted value to a prediction mode indicated by a prediction mode value from "0" to "M-1" and selects a prediction mode. The prediction unit 3403 outputs the selected prediction mode, specifically, a prediction mode value indicating the prediction mode, to the arithmetic coding unit. The prediction unit 3403 performs, for example, the process of step S3455.

[0577] The prediction residual calculation unit 3404 calculates (generates) a prediction residual of the predicted value of the attribute information generated by the prediction unit 3403. The prediction residual calculation unit 3404 performs the process of step S3456.

[0578] The quantization unit 3405 quantizes the prediction residual of the attribute information calculated by the prediction residual calculation unit 3404 .

[0579] The arithmetic coding unit 3406 arithmetically codes the prediction residuals after being quantized by the quantization unit 3405. The arithmetic coding unit 3406 outputs a bitstream including the arithmetically coded prediction residuals to, for example, a three-dimensional data decoding device.

[0580] Note that the prediction residual may be binarized by, for example, the quantization unit 3405 before being arithmetically coded by the arithmetic coding unit 3406. Note that the arithmetic coding unit 3406 may generate and code various header information. Furthermore, the arithmetic coding unit 3406 may obtain the prediction mode used for coding from the prediction block, arithmetically code it, and add it to the bitstream.

[0581] The inverse quantization unit 3407 inverse quantizes the prediction residuals after being quantized by the quantization unit 3405. The inverse quantization unit 3407 performs the process of step S3459.

[0582] The decoded value generation unit 3408 generates a decoded value by adding the predicted value of the attribute information generated by the prediction unit 3403 and the prediction residual after inverse quantization by the inverse quantization unit 3407.

[0583] The memory 3409 is a memory that stores the decoded values ​​of the attribute information of each 3D point decoded by the decoded value generation unit 3408. For example, when generating a predicted value of a 3D point that has not yet been coded, the prediction unit 3403 generates the predicted value by using the decoded values ​​of the attribute information of each 3D point stored in the memory 3409.

[0584] Fig. 101 is a block diagram showing the configuration of the attribute information decoding unit 3410 included in the three-dimensional data decoding device according to embodiment 5. Fig. 101 shows details of the attribute information decoding unit, which is one of the position information decoding unit and the attribute information decoding unit included in the three-dimensional data decoding device.

[0585] The attribute information decoding unit 3410 includes an LoD generation unit 3411 , a surrounding search unit 3412 , a prediction unit 3413 , an arithmetic decoding unit 3414 , an inverse quantization unit 3415 , a decoded value generation unit 3416 , and a memory 3417 .

[0586] The LoD generation unit 3411 generates LoD using the position information (geometry information) of the three-dimensional points decoded by a position information decoding unit (not shown).

[0587] The surrounding search unit 3412 searches for nearby 3D points adjacent to each 3D point using the LoD generation result by the LoD generation unit 3411 and distance information indicating the distance between each 3D point.

[0588] The prediction unit 3413 generates a predicted value of the attribute information of the target 3D point to be decoded. The prediction unit 3413 performs the process of step S3485, for example.

[0589] The arithmetic decoding unit 3414 arithmetically decodes prediction residuals in the bitstream acquired from the attribute information encoding unit 3400. The arithmetic decoding unit 3414 may decode various types of header information. Furthermore, the arithmetic decoding unit 3414 may output the arithmetically decoded prediction mode to the prediction unit 3413. In this case, the prediction unit 3413 may calculate a prediction value using the prediction mode obtained by arithmetically decoding in the arithmetic decoding unit 3414.

[0590] The inverse quantization unit 3415 inverse quantizes the prediction residuals arithmetically decoded by the arithmetic decoding unit 3414 .

[0591] The decoded value generation unit 3416 generates a decoded value by adding the predicted value generated by the prediction unit 3413 and the prediction residual after inverse quantization by the inverse quantization unit 3415. The decoded value generation unit 3416 outputs the decoded attribute information data to another device.

[0592] The memory 3417 is a memory that stores the decoded values ​​of the attribute information of each 3D point decoded by the decoded value generation unit 3416. For example, when generating a predicted value of a 3D point that has not yet been decoded, the prediction unit 3413 generates the predicted value by using the decoded value of the attribute information of each 3D point stored in the memory 3417.

[0593] (Sixth embodiment) As described above, the three-dimensional data encoding device calculates the maximum absolute difference value of attribute values ​​at N three-dimensional points (surrounding three-dimensional points) surrounding a target three-dimensional point to be encoded, which can be used for prediction by both the three-dimensional data encoding device and the three-dimensional data decoding device (i.e., the maximum absolute value of the differences in attribute values ​​between any two three-dimensional points among the N three-dimensional points).The three-dimensional data encoding device also fixes the prediction mode according to the calculated value, i.e., switches between using a predetermined arbitrary prediction mode or selecting a prediction mode and adding it to the bitstream.

[0594] However, the three-dimensional data encoding device does not have to switch between fixing the prediction mode or selecting a prediction mode and adding it to the bitstream according to the maximum absolute difference value. For example, the three-dimensional data encoding device may select whether to fix the prediction mode or select a prediction mode based on the above-mentioned conditions, and add the result to the bitstream as a prediction mode fixing flag.

[0595] This allows the three-dimensional data decoding device to determine whether the three-dimensional data encoding device has fixed the prediction mode or selected a prediction mode and encoded it by decoding the prediction mode fixing flag attached to the bitstream.

[0596] Then, if the prediction mode is fixed by the 3D data encoding device, the 3D data decoding device can determine that the prediction mode is not encoded in the bitstream. On the other hand, if the prediction mode is selected by the 3D data encoding device, the 3D data decoding device can determine that the prediction mode of the bitstream needs to be decoded. This allows the 3D data decoding device to correctly decode the encoded prediction mode included in the bitstream.

[0597] This also allows the 3D data decoding device to arithmetically decode the prediction mode without calculating the maximum absolute difference value of attribute values ​​at N 3D points surrounding the target 3D point to be coded, which can be used for prediction. This allows the 3D data decoding device to perform arithmetic decoding of the bitstream and LoD generation, etc. in parallel. Therefore, the 3D data decoding device can improve the overall processing throughput.

[0598] The three-dimensional data encoding device may add a prediction mode fixing flag to each three-dimensional point.

[0599] This allows the 3D data decoding device to switch between fixing and selecting a prediction mode for each 3D point based on the prediction mode fixing flag, thereby improving the coding efficiency of the 3D data encoding device.

[0600] The three-dimensional data encoding device may set a prediction mode fixing flag for each LoD. For example, the three-dimensional data encoding device sets the prediction mode fixing flag to 0 for an upper layer of each LoD where prediction is less likely to be accurate, allowing the prediction mode to be selected, and sets the prediction mode fixing flag to 1 for a lower layer where prediction is more likely to be accurate, thereby fixing the prediction mode. This allows the three-dimensional data encoding device to reduce the amount of code added for the prediction mode.

[0601] FIG. 102 is a flowchart showing the process of determining a prediction mode executed by the three-dimensional data encoding device according to this embodiment.

[0602] First, the three-dimensional data encoding device calculates the maximum absolute difference value maxdiff of the attribute values ​​at N three-dimensional points around the target three-dimensional point to be encoded (S3501).

[0603] Fig. 103 is a diagram showing an example of syntax for the process of determining a prediction mode executed by the three-dimensional data encoding device according to this embodiment. Specifically, Fig. 103 is a diagram showing an example of syntax for step S3501 in Fig. 102.

[0604] 103, the attribute values ​​around the target 3D point to be coded are indicated by a[0] to a[N-1], and the maximum absolute difference value is indicated by maxdiff. Here, the N attribute values ​​around the target 3D point to be coded are attribute values ​​coded by the 3D data coding device and decoded by the 3D data decoding device.

[0605] The three-dimensional data encoding device calculates the maximum absolute difference value maxdiff, for example, using the example syntax shown in FIG.

[0606] The N three-dimensional points around the target three-dimensional point to be encoded that are used for prediction by the three-dimensional data encoding device are N coded and decoded three-dimensional points whose distance from the target three-dimensional point to be encoded is less than the threshold value THd.

[0607] Here, as described above, the three-dimensional data encoding device may add the maximum value of N as NumNeighborPoint to the bit stream. Note that when the number of encoded and decoded three-dimensional points in the attribute information of the three-dimensional points around the target three-dimensional points to be encoded is less than the value of NumNeighborPoint, etc., the value of N does not have to match the value of NumNeighborPoint.

[0608] Referring to FIG. 102 again, next, the three-dimensional data encoding device determines whether maxdiff < Thfix is satisfied (S3502). Note that Thfix is a constant arbitrarily determined in advance.

[0609] When the three-dimensional data encoding device determines that maxdiff < Thfix is satisfied (Yes in S3502), it sets the prediction mode fixed flag (fixedPredMode) to 1 and performs arithmetic encoding (S3503).

[0610] Next, the three-dimensional data encoding device determines that the value of the prediction mode (PredMode / hereinafter also simply referred to as the prediction mode) is 0 (S3504).

[0611] On the other hand, when the three-dimensional data encoding device determines that maxdiff < Thfix is not satisfied (No in S3502), it sets the prediction mode fixed flag to 0 and performs arithmetic encoding (S3505).

[0612] Next, the three-dimensional data encoding device selects a prediction mode (S3506).

[0613] Next, the three-dimensional data encoding device performs arithmetic encoding on the selected prediction mode (S3507).

[0614] In the above example, the three-dimensional data device fixes the prediction mode to 0 if the maximum absolute difference value of the attribute values ​​of three-dimensional points surrounding the target three-dimensional point to be coded, which is used for prediction, is smaller than Thfix[i], but this is not necessarily limited to this. For example, the three-dimensional data coding device may fix the prediction mode to any one of 0 to M-1.

[0615] Furthermore, the three-dimensional data encoding device may add a fixed prediction mode value (also called PredMode or mode number) to the bitstream.

[0616] FIG. 104 is a flowchart showing the process of determining a prediction mode executed by the three-dimensional data decoding device according to this embodiment.

[0617] First, the three-dimensional data decoding device arithmetically decodes the coded prediction mode fixing flag included in the bitstream (S3511).

[0618] Next, the three-dimensional data decoding device determines whether or not the prediction mode fixing flag==1 is satisfied (S3512).

[0619] When it is determined that the prediction mode fixing flag==1 is satisfied (Yes in S3512), the three-dimensional data decoding device determines the value of the prediction mode to be 0 (S3513).

[0620] On the other hand, if it is determined that the prediction mode fixing flag==1 is not satisfied (No in S3512), the three-dimensional data decoding device determines the value of the prediction mode by decoding it from the bitstream (S3514).

[0621] The three-dimensional data decoding device determines a predicted value based on the determined prediction mode.

[0622] The prediction mode fixing flag may be provided at any position.

[0623] FIG. 105 is a diagram showing an example of the syntax of attribute data when a prediction mode fixing flag is provided for each 3D point.

[0624] fixedPredMode is a flag indicating whether the 3D data encoding device fixes the prediction mode. For example, if the value of fixedPredMode is 1, the 3D data encoding device fixes the prediction mode, and if the value of fixedPredMode is 0, the 3D data encoding device may select a prediction mode.

[0625] Furthermore, fixedPredMode may be set for each LoD layer as shown in FIG.

[0626] FIG. 106 is a diagram illustrating an example of the syntax of attribute data in a case where a prediction mode fixing flag is provided for each layer of LoD.

[0627] fixedPredMode[i] is a flag indicating whether the prediction mode of the i-th layer of the LoD is fixed.

[0628] PredMode is a value indicating a prediction mode for encoding and decoding the attribute value of the j-th 3D point of layer i. PredMode takes on values ​​from 0 to M-1 (where M is the total number of prediction modes). Note that the 3D data decoding device may estimate PredMode to be 0 if PredMode is not present in the bitstream (specifically, if !fixedPredMode&&NumPredMode[i]>1 is not satisfied).

[0629] Furthermore, if PredMode is not present in the bitstream, the three-dimensional data decoding device does not need to set PredMode to 0, and may adopt any value between 0 and M-1 as an estimated value.

[0630] Furthermore, the three-dimensional data encoding device may add an estimated value to a separate header or the like when PredMode is not present in the bitstream.

[0631] Furthermore, as described above, the three-dimensional data encoding device may binarize PredMode with a truncated unary code using the value of the total number M of prediction modes, and arithmetically encode the binarized value.

[0632] It should be noted that truncated unary code is one of the binarization techniques. In truncated unary code, for a multi-value signal that takes a value other than the maximum value, a signal is generated in which the same number of 1s as the value indicated by the multi-value signal are arranged and a 0 is added at the end. For a multi-value signal that takes the maximum value, the maximum value is set in advance, and a signal is generated in which the same number of 1s as the value indicated by the multi-value signal are arranged (a signal without a 0 at the end). For example, as described above, if there are five prediction modes, when the three-dimensional data encoding device binarizes each prediction mode with truncated unary code, the respective prediction modes become 0, 10, 110, 1110, and 1111 in order from prediction mode 0 to 4. In this way, the three-dimensional data encoding device binarizes the prediction modes with truncated unary code using the number of prediction modes (i.e., according to the number of prediction modes).

[0633] In a truncated unary code, for a multi-level signal that takes a value other than the maximum value, a signal may be generated in which the same number of 0s as the value indicated by the multi-level signal are arranged, with a 1 added at the end. In other words, the above-mentioned 0s and 1s may be reversed.

[0634] Furthermore, the three-dimensional data encoding device may arithmetically encode the value of the total number M of prediction modes as NumPredMode and add it to the header.

[0635] It should be noted that NumPredMode is a value indicating the total number of prediction modes.

[0636] This allows the 3D data decoding device to decode NumPredMode in the header to calculate the total number M of prediction modes, and decode PredMode using the total number M of prediction modes. Therefore, the 3D data decoding device can set (generate) LoD, calculate 3D points around the target 3D point to be decoded that can be used for prediction, and perform arithmetic decoding of the bitstream without waiting for the calculation of the number of prediction modes to which prediction values ​​are assigned. This allows the 3D data decoding device to perform arithmetic decoding of the bitstream and LoD generation, etc. in parallel, thereby improving the overall processing throughput.

[0637] Furthermore, the n-bit code is coded data of the prediction residual of the attribute information value. The bit length of the n-bit code depends on the value of R_TH[i]. For example, if the value indicated by R_TH[i] is 63, the n-bit code is 6 bits long, and if the value indicated by R_TH[i] is 255, the n-bit code is 8 bits long. The value indicated by R_TH may be arbitrarily determined in advance.

[0638] Furthermore, the remaining code is coded data of the prediction residual of the attribute information value that is coded using the exponential-Golomb code. The three-dimensional data decoding device decodes the remaining code when the n-bit code is the same as R_TH[i], and adds the value of the n-bit code and the value of the remaining code to decode the prediction residual.

[0639] It should be noted that the three-dimensional data decoding device does not need to decode the remaining code if the n-bit code does not have the same value as R_TH[i].

[0640] FIG. 107 is a flowchart showing an example of the prediction mode encoding process of the three-dimensional data encoding device.

[0641] First, the three-dimensional data encoding device binarizes the prediction mode with a truncated unary code using the total number M of prediction modes (S3521).

[0642] Next, the three-dimensional data encoding device arithmetically encodes the binarized data of the truncated unary code (S3522).

[0643] Next, the three-dimensional data encoding device adds the total number M of prediction modes as NumPredMode to the header and performs encoding (S3523).

[0644] The three-dimensional data encoding device transmits, for example, a bitstream including the encoded NumPredMode to the three-dimensional data decoding device.

[0645] FIG. 108 is a flowchart showing an example of a prediction mode decoding process in the three-dimensional data decoding device.

[0646] First, the three-dimensional data decoding device decodes the coded NumPredMode included in the bitstream and sets the total number M of prediction modes (S3524).

[0647] Next, the three-dimensional data decoding device arithmetically decodes the coded PredMode using the decoded total number M of prediction modes, and generates binarized data of a truncated unary code (S3525).

[0648] Next, the three-dimensional data decoding device calculates a prediction mode from the binarized data of the truncated unary code (S3526).

[0649] Furthermore, the three-dimensional data encoding device may perform arithmetic encoding of the above-mentioned fixedPredMode using an encoding table. Note that each three-dimensional data encoding device may update the occurrence probabilities of 0 and 1 in the encoding table according to the value of fixedPredMode that has actually occurred. Furthermore, the three-dimensional data encoding device may fix the occurrence probabilities of 0 and 1 in one of the encoding tables. This allows the three-dimensional data encoding device to reduce the number of updates to the occurrence probabilities of 0 and 1 in the encoding table, thereby reducing the amount of processing.

[0650] FIG. 109 is a flowchart of three-dimensional data encoding processing by the three-dimensional data encoding device according to this embodiment.

[0651] First, the three-dimensional data encoding device encodes position information (geometry) (S3531). Note that here, the three-dimensional data encoding device may use, for example, an octree representation for encoding.

[0652] Next, if the position of the 3D point changes due to quantization or the like after encoding the position information, the 3D data encoding device reassigns the attribute information of the original 3D point to the changed 3D point (S3532).

[0653] The three-dimensional data encoding device may interpolate and reassign the values ​​of the attribute information according to the amount of change in position. For example, the three-dimensional data encoding device may detect N three-dimensional points surrounding the target three-dimensional point to be encoded before the change that is close to the three-dimensional position after the change, and weight the values ​​of the attribute information of the detected N three-dimensional points based on the distance from the target three-dimensional position to be encoded after the change to each of the N three-dimensional points, and use this value as the value of the attribute information of the three-dimensional point after the change. Furthermore, when two or more three-dimensional points are changed to the same three-dimensional position by quantization or the like, the three-dimensional data encoding device may assign the average value of the attribute information of the two or more three-dimensional points before the change as the value of the attribute information after the change.

[0654] Next, the three-dimensional data encoding device encodes the reallocated attribute information (Attribute) (S3533). For example, when encoding a plurality of pieces of attribute information as attribute information, the three-dimensional data encoding device may encode them in order.

[0655] Furthermore, for example, when the three-dimensional data encoding device encodes color and reflectance as attribute information, it may generate a bit stream in which the encoding result of reflectance is added after the encoding result of color.

[0656] The three-dimensional data encoding device may add the encoded results of the attribute information to the bitstream in any order. The three-dimensional data encoding device may also add the start location of the encoded data for each attribute information in the bitstream to a header or the like.

[0657] This allows the three-dimensional data decoding device to decode attribute information that requires decoding, thereby reducing the amount of processing by omitting the decoding process for attribute information that does not require decoding.

[0658] Furthermore, the three-dimensional data encoding device may encode multiple pieces of attribute information in parallel and integrate the encoding results into one bitstream.

[0659] This allows the three-dimensional data encoding device to encode multiple pieces of attribute information at high speed.

[0660] FIG. 110 is a flowchart of the attribute information encoding process (S3533) shown in FIG.

[0661] First, the three-dimensional data encoding device sets the LoD (S35331). That is, the three-dimensional data encoding device assigns each three-dimensional point to one of a plurality of LoDs.

[0662] Next, the three-dimensional data encoding device starts a loop for each LoD (S35332). That is, the three-dimensional data encoding device repeats the processing of steps S35333 to S35341 for each LoD.

[0663] Next, the three-dimensional data encoding device starts a loop for each three-dimensional point (S35333). That is, the three-dimensional data encoding device repeatedly performs the processes of steps S35334 to S35340 for each three-dimensional point for a certain LoD. Note that Fig. 110 shows encoding of a target three-dimensional point P to be encoded.

[0664] Next, the three-dimensional data encoding device searches for a plurality of surrounding points, which are three-dimensional points existing around the target three-dimensional point P, to be used in calculating a predicted value of the target three-dimensional point P to be processed (S35334).

[0665] Next, the three-dimensional data encoding device calculates a predicted value of the target three-dimensional point P (S35335).

[0666] Next, the three-dimensional data encoding device calculates a prediction residual, which is the difference between the attribute information of the target three-dimensional point P and the predicted value (S35336).

[0667] Next, the three-dimensional data encoding device calculates a quantized value by quantizing the prediction residual (S35337).

[0668] Next, the three-dimensional data encoding device arithmetically encodes the quantized value (S35338).

[0669] Next, the three-dimensional data encoding device calculates an inverse quantized value by inverse quantizing the quantized value (S35339).

[0670] Next, the three-dimensional data encoding device generates a decoded value by adding the predicted value to the inverse quantized value (S35340).

[0671] Next, the three-dimensional data encoding device ends the loop for each three-dimensional point (S35341).

[0672] The three-dimensional data encoding device also ends the loop for each LoD (S35342).

[0673] FIG. 111 is a flowchart of the calculation process (S35335) of the predicted value shown in FIG.

[0674] First, the 3D data encoding device calculates a weighted average value of attribute values ​​of N surrounding 3D points around a target 3D point to be encoded that can be used for prediction, and assigns the weighted average value to prediction mode 0 (S353351).

[0675] Next, the three-dimensional data encoding device calculates the maximum absolute difference value maxdiff of the attribute values of the N surrounding three-dimensional points of the target three-dimensional point to be encoded (S353352).

[0676] Next, the three-dimensional data encoding device determines whether the maximum absolute difference value maxdiff < Thfix is satisfied (S353353).

[0677] When the three-dimensional data encoding device determines that the maximum absolute difference value maxdiff < Thfix is satisfied (Yes in S353353), it sets the prediction mode fixed flag to 1 and performs arithmetic encoding (S353354).

[0678] Next, the three-dimensional data encoding device determines the prediction mode to 0 (the prediction mode indicating the weighted average value as the predicted value) (S353355).

[0679] Next, the three-dimensional data encoding device arithmetic-encodes the predicted value of the determined prediction mode and outputs it to, for example, the three-dimensional data decoding device (S353356).

[0680] On the other hand, when the three-dimensional data encoding device determines that the maximum absolute difference value maxdiff < Thfix is not satisfied (No in S353353), it sets the prediction mode fixed flag to 0 and performs arithmetic encoding (S353357).

[0681] Next, the three-dimensional data encoding device selects and determines the prediction mode (S353358).

[0682] Next, the three-dimensional data encoding device arithmetic-encodes the selected and determined prediction mode (S353359).

[0683] Note that, as described above, the three-dimensional data encoding device may binarize and arithmetic-encode the prediction mode PredMode using the total number of prediction modes M in a truncated unary code. Further, the three-dimensional data encoding device may encode and add the total number of prediction modes M as NumPredMode to the header.

[0684] This allows the three-dimensional data decoding device to correctly decode the prediction mode PredMode by decoding NumPredMode in the header.

[0685] Note that the three-dimensional data encoding device does not need to encode PredMode when NumPredMode=1, thereby enabling the three-dimensional data encoding device to reduce the amount of code when NumPredMode=1.

[0686] FIG. 112 is a flowchart of the prediction mode selection process (S353358) shown in FIG.

[0687] First, the three-dimensional data encoding device assigns prediction modes 1 to N to attribute information of N three-dimensional points surrounding a target three-dimensional point to be encoded, in order of closest distance to the target three-dimensional point (S3541). For example, the three-dimensional data encoding device generates N+1 prediction modes. Note that if N+1 exceeds the total number M (NumPredMode) of prediction modes to be added to the bitstream, the three-dimensional data encoding device may generate up to M prediction modes.

[0688] Next, the three-dimensional data encoding apparatus calculates the cost of each mode and selects the prediction mode with the smallest cost (S3542).

[0689] FIG. 113 is a flowchart showing a specific example of the prediction mode selection process (S3542) shown in FIG.

[0690] First, the three-dimensional data encoding device performs i=0 and mincost=∞ (S35421).

[0691] Next, the three-dimensional data encoding device calculates the cost (cost[i]) of the i-th prediction mode PredMode[i] (S35422).

[0692] Next, the three-dimensional data encoding device determines whether cost[i] < mincost is satisfied (S35423).

[0693] Next, when the three-dimensional data encoding device determines that cost[i] < mincost is satisfied (Yes in S35423), it sets mincost = cost[i] and sets the prediction mode to PredMode[i] (S35424).

[0694] Next, after step S35424 or when the three-dimensional data encoding device determines that cost[i] < mincost is not satisfied (No in S35423), it performs i = i + 1 (S35425).

[0695] Next, the three-dimensional data encoding device determines whether i < the number of prediction modes (total number of prediction modes) is satisfied (S35426).

[0696] When the three-dimensional data encoding device determines that i < the number of prediction modes is not satisfied (No in S35426), it ends the selection process, and when it determines that i < the number of prediction modes is satisfied (Yes in S35426), it returns the process to step S35422.

[0697] FIG. 114 is a flowchart of three-dimensional data decoding processing by the three-dimensional data decoding device according to the present embodiment.

[0698] The three-dimensional data decoding device decodes the position information (geometry) of the encoded three-dimensional points (S3551). The three-dimensional data decoding device may decode the position information using, for example, an octree representation.

[0699] Next, the three-dimensional data decoding device decodes the attribute information of the encoded three-dimensional points (S3552).

[0700] When decoding a plurality of pieces of attribute information as attribute information, the three-dimensional data decoding device may decode them in order. For example, when decoding color and reflectance as attribute information, the three-dimensional data decoding device may decode a bit stream in which the encoding result of reflectance is added after the encoding result of color in that order.

[0701] Furthermore, the three-dimensional data decoding device may decode the encoded results of the attribute information added to the bitstream in any order.

[0702] Furth...

Claims

1. 1. A three-dimensional data encoding method for encoding three-dimensional points having a predictive tree structure, comprising: encoding the node's location based on a method corresponding to one of one or more selectable methods used to predict the node's location; generating a bitstream including mode information indicating the method used for said node; the one or more selectable methods are varied based on a depth of the node in the prediction tree structure; When the depth of the node in the prediction tree structure is 1, the number of the one or more selectable methods is 2. Three-dimensional data encoding method.

2. the mode information includes a value indicating a method used by the node; the value is less than or equal to the depth value of the node in the predictive tree structure. The three-dimensional data encoding method according to claim 1 .

3. In the encoding, a predicted value of the node is calculated based on the method. The three-dimensional data encoding method according to claim 1 .

4. In the encoding, a prediction residual indicating a difference between the position of the node and the calculated predicted value of the node is derived; The three-dimensional data encoding method according to claim 3.

5. If the depth of the node in the prediction tree structure is 0, the number of the one or more selectable methods is 1. The three-dimensional data encoding method according to claim 3.

6. If the depth of the node in the prediction tree structure is two, the number of the one or more selectable ways is three; or If the depth of the node in the prediction tree structure is 3, the number of the one or more selectable methods is 4. The three-dimensional data encoding method according to claim 1 .

7. the one or more selectable methods include a first method used for a root node of the prediction tree structure; 7. The three-dimensional data encoding method according to claim 6.

8. The one or more selectable methods include a second method in which a node having a parent node in the prediction tree structure can be used, a third method in which a node having a parent node and a grandparent node can be used, and a fourth method in which a node having a parent node, a grandparent node, and a great-grandparent node can be used.

7. The three-dimensional data encoding method according to claim 6.

9. 1. A three-dimensional data decoding method for decoding three-dimensional points having a predictive tree structure, comprising: obtaining mode information indicating a method used to predict the node's position in the prediction tree structure; Decoding the node's location based on the mode information; the method indicated by the mode information corresponds to one of one or more selectable methods that change based on a depth of the node in the prediction tree structure; When the depth of the node in the prediction tree structure is 1, the number of the one or more selectable methods is 2. Three-dimensional data decoding method.

10. the mode information includes a value corresponding to one of the one or more selectable methods, the value being less than or equal to a depth value of the node in a prediction tree structure; The three-dimensional data decoding method according to claim 9.

11. In the decoding, a predicted value to be used for calculating the position of the node is calculated based on the method indicated by the mode information. The three-dimensional data decoding method according to claim 9.

12. deriving a prediction residual during the decoding, and calculating a position of the node using the prediction and the prediction residual; The three-dimensional data decoding method according to claim 11.

13. If the depth of the node in the prediction tree structure is 0, the number of the one or more selectable methods is 1. The three-dimensional data decoding method according to claim 11.

14. If the depth of the node in the prediction tree structure is two, the number of the one or more selectable ways is three; or If the depth of the node in the prediction tree structure is 3, the number of the one or more selectable methods is 4. The three-dimensional data decoding method according to claim 9.

15. the one or more selectable methods include a first method used for a root node of the prediction tree structure; The three-dimensional data decoding method according to claim 14.

16. The one or more selectable methods include a second method in which a node having a parent node in the prediction tree structure can be used, a third method in which a node having a parent node and a grandparent node can be used, and a fourth method in which a node having a parent node, a grandparent node, and a great-grandparent node can be used. The three-dimensional data decoding method according to claim 14.

17. A three-dimensional data encoding device for encoding three-dimensional points having a predictive tree structure, comprising: a processor; a memory; The processor uses the memory to: encoding the node's location based on a method corresponding to one of one or more selectable methods used to predict the node's location; generating a bitstream including mode information indicating the method used for said node; the one or more selectable methods are varied based on a depth of the node in the prediction tree structure; When the depth of the node in the prediction tree structure is 1, the number of the one or more selectable methods is 2. Three-dimensional data encoding device.

18. A three-dimensional data decoding device for decoding three-dimensional points having a predictive tree structure, comprising: a processor; a memory; The processor uses the memory to: obtaining mode information indicating a method used to predict the node's position in the prediction tree structure; Decoding the node's location based on the mode information; the method indicated by the mode information corresponds to one of one or more selectable methods that change based on a depth of the node in the prediction tree structure; When the depth of the node in the prediction tree structure is 1, the number of the one or more selectable methods is 2. Three-dimensional data decoding device.

Images