Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

Abstract

A three-dimensional data encoding method encodes a plurality of attribute information having each of a plurality of three-dimensional points using parameters (S6701), generates a bit stream including the encoded plurality of attribute information, control information, and a plurality of first attribute control information (S6702), wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a different type of attribute information, and the plurality of first attribute control information each corresponds to the plurality of attribute information and each of the plurality of first attribute control information includes a first identification information indicating that it corresponds to one of the plurality of type information.

Description

Technology Field

[0001] The present disclosure relates to a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, and a three-dimensional data decoding device. Background Technology

[0002] In a wide range of fields, such as computer vision for the autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, or image transmission, the widespread adoption of devices or services utilizing three-dimensional data is expected in the future. Three-dimensional data is acquired through various methods, including distance sensors such as rangefinders, stereo cameras, or combinations of multiple monocular cameras.

[0003] One method of representing three-dimensional data is called a point cloud, which represents the shape of a three-dimensional structure using a point cloud within three-dimensional space. In a point cloud, the location and color of the point cloud are stored. While point clouds are expected to become the mainstream method for representing three-dimensional data, the amount of data from a point cloud is very large. Therefore, for the storage or transmission of three-dimensional data, compression of the data volume through encoding becomes essential, just as with two-dimensional video (for example, MPEG-4 AVC or HEVC standardized by MPEG).

[0004] In addition, point cloud compression is partially supported by open libraries (Point Cloud Libraries) that perform point cloud-related processing.

[0005] In addition, a technology is known for searching for and displaying facilities located around a vehicle using three-dimensional map data (see, for example, Patent Document 1). Prior art literature

[0006] International Publication No. 2014 / 020663 The problem to be solved

[0007] In the encoding and decoding of three-dimensional data, it is desired to be able to correctly decode attribute information of three-dimensional points.

[0008] The present disclosure aims to provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device capable of correctly decoding attribute information of a three-dimensional point. means of solving the problem

[0009] A three-dimensional data encoding method according to one aspect of the present disclosure encodes a plurality of attribute information having each of a plurality of three-dimensional points using parameters, and generates a bit stream comprising the encoded plurality of attribute information, control information, and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a different type of attribute information, and the plurality of first attribute control information each corresponds to the plurality of attribute information and each of the plurality of first attribute control information includes a first identification information indicating that it corresponds to any one of the plurality of type information.

[0010] A three-dimensional data decoding method according to one aspect of the present disclosure comprises acquiring a plurality of encoded attribute information and a parameter by acquiring a bit stream and decoding the encoded plurality of attribute information using the parameter to decode the plurality of attribute information having each of the plurality of three-dimensional points, wherein the bit stream includes control information and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a different type of attribute information, wherein the plurality of first attribute control information each corresponds to the plurality of attribute information and each of the plurality of first attribute control information includes a first identification information indicating that it corresponds to any one of the plurality of type information. Effects of the invention

[0011] The present disclosure may provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device capable of correctly decoding attribute information of a three-dimensional point. Brief explanation of the drawing

[0012] FIG. 1 is a diagram showing the configuration of a three-dimensional data encoding and decoding system according to Embodiment 1. FIG. 2 is a diagram showing an example of the configuration of point cloud data according to Embodiment 1. FIG. 3 is a diagram showing an example of the configuration of a data file in which point cloud data information according to Embodiment 1 is described. FIG. 4 is a diagram showing the types of point cloud data according to Embodiment 1. FIG. 5 is a diagram showing the configuration of the first encoding unit according to embodiment 1. FIG. 6 is a block diagram of the first encoding unit according to embodiment 1. FIG. 7 is a diagram showing the configuration of the first decoding unit according to embodiment 1. FIG. 8 is a block diagram of the first decoding unit according to embodiment 1. FIG. 9 is a diagram showing the configuration of the second encoding unit according to embodiment 1. FIG. 10 is a block diagram of the second encoding unit according to embodiment 1. FIG. 11 is a drawing showing the configuration of the second decoding unit according to embodiment 1. FIG. 12 is a block diagram of the second decoding unit according to embodiment 1. FIG. 13 is a diagram showing a protocol stack related to PCC encoded data according to Embodiment 1. FIG. 14 is a block diagram of an encoding unit according to embodiment 1. FIG. 15 is a block diagram of a decoding unit according to embodiment 1. FIG. 16 is a flowchart of encoding processing according to Embodiment 1. FIG. 17 is a flowchart of decoding processing according to embodiment 1. FIG. 18 is a drawing showing the basic structure of an ISOBMFF according to embodiment 2. FIG. 19 is a diagram showing a protocol stack according to embodiment 2. FIG. 20 is a diagram showing an example of saving a NAL unit according to embodiment 2 to a file for codec 1. FIG. 21 is a diagram showing an example of saving a NAL unit according to embodiment 2 to a file for codec 2. FIG. 22 is a drawing showing the configuration of the first multiplexer according to embodiment 2. FIG. 23 is a diagram showing the configuration of the first demultiplexer according to embodiment 2. FIG. 24 is a drawing showing the configuration of the second multiplexer according to embodiment 2. FIG. 25 is a diagram showing the configuration of a second demultiplexer according to embodiment 2. FIG. 26 is a flowchart of processing by the first multiplexer according to embodiment 2. FIG. 27 is a flowchart of processing by the second multiplexer according to embodiment 2. FIG. 28 is a flowchart of processing by the first demultiplexer and the first decoding unit according to embodiment 2. FIG. 29 is a flowchart of processing by the second demultiplexer and the second decoding unit according to embodiment 2. FIG. 30 is a diagram showing the configuration of an encoding unit and a third multiplexing unit according to embodiment 3. FIG. 31 is a diagram showing the configuration of the third demultiplexer and decoding unit according to embodiment 3. FIG. 32 is a flowchart of processing by the third multiplexer according to embodiment 3. FIG. 33 is a flowchart of processing by the third demultiplexer and decoding unit according to embodiment 3. FIG. 34 is a flowchart of processing by a three-dimensional data storage device according to embodiment 3. FIG. 35 is a flowchart of processing by a three-dimensional data acquisition device according to embodiment 3. FIG. 36 is a diagram showing the configuration of the encoding unit and the multiplexing unit according to embodiment 4. FIG. 37 is a diagram showing an example of the configuration of encoded data according to embodiment 4. FIG. 38 is a diagram showing an example of the configuration of encoding data and a NAL unit according to embodiment 4. FIG. 39 is a drawing showing an example of semantics of the pcc_nal_unit_type according to embodiment 4. FIG. 40 is a diagram showing an example of the transmission sequence of a NAL unit according to embodiment 4. FIG. 41 is a flowchart of processing by a three-dimensional data encoding device according to embodiment 4. FIG. 42 is a flowchart of processing by a three-dimensional data decoding device according to embodiment 4. FIG. 43 is a flowchart of multiplexing processing according to embodiment 4. FIG. 44 is a flowchart of the demultiplexing process according to Embodiment 4. FIG. 45 is a flowchart of processing by a three-dimensional data encoding device according to embodiment 4. FIG. 46 is a flowchart of processing by a three-dimensional data decoding device according to embodiment 4. FIG. 47 is a block diagram of a divided section according to embodiment 5. FIG. 48 is a drawing showing an example of dividing a slice and a tile according to embodiment 5. FIG. 49 is a drawing showing an example of a division pattern of slices and tiles according to embodiment 5. FIG. 50 is a block diagram of the first decoding unit according to embodiment 6. FIG. 51 is a block diagram of the first decoding unit according to embodiment 6. FIG. 52 is a drawing showing an example of the shape of a tile according to embodiment 6. FIG. 53 is a drawing showing an example of a tile and a slice according to embodiment 6. FIG. 54 is a block diagram of a divided section according to embodiment 6. FIG. 55 is a drawing showing an example of a map viewed from the top surface of point cloud data according to embodiment 6. FIG. 56 is a drawing showing an example of tile division according to embodiment 6. FIG. 57 is a drawing showing an example of tile division according to embodiment 6. FIG. 58 is a drawing showing an example of tile division according to embodiment 6. FIG. 59 is a diagram showing an example of tile data stored on a server according to embodiment 6. FIG. 60 is a drawing showing a system for tile division according to embodiment 6. FIG. 61 is a drawing showing an example of slice division according to embodiment 6. FIG. 62 is a drawing showing an example of a dependency relationship according to Embodiment 6. FIG. 63 is a diagram showing an example of the decoding sequence of data according to Embodiment 6. FIG. 64 is a diagram showing an example of encoding data of a tile according to embodiment 6. FIG. 65 is a block diagram of a joint portion according to embodiment 6. FIG. 66 is a diagram showing an example of the configuration of encoding data and a NAL unit according to embodiment 6. FIG. 67 is a flowchart of encoding processing according to embodiment 6. FIG. 68 is a flowchart of decoding processing according to embodiment 6. FIG. 69 is a drawing showing an example of the syntax of tile additional information according to embodiment 6. FIG. 70 is a block diagram of an encoding and decoding system according to embodiment 6. FIG. 71 is a drawing showing an example of the syntax of slice addition information according to embodiment 6. FIG. 72 is a flowchart of encoding processing according to embodiment 6. FIG. 73 is a flowchart of decoding processing according to embodiment 6. FIG. 74 is a flowchart of encoding processing according to embodiment 6. FIG. 75 is a flowchart of decoding processing according to embodiment 6. FIG. 76 is a flowchart showing the re-initialization process of the CABAC encoding / decoding engine according to the CABAC initialization flag during encoding or decoding according to Embodiment 7. FIG. 77 is a block diagram showing the configuration of a first encoding unit included in a three-dimensional data encoding device according to embodiment 7. FIG. 78 is a block diagram showing the configuration of a divided section according to embodiment 7. FIG. 79 is a block diagram showing the configuration of a position information encoding unit and an attribute information encoding unit according to embodiment 7. FIG. 80 is a block diagram showing the configuration of the first decoding unit according to embodiment 7. FIG. 81 is a block diagram showing the configuration of a position information decoding unit and an attribute information decoding unit according to embodiment 7. FIG. 82 is a flowchart showing an example of processing regarding the initialization of CABAC in the encoding of position information or attribute information according to Embodiment 7. FIG. 83 is a diagram showing an example of CABAC initialization timing in point cloud data as a bit stream according to Embodiment 7. FIG. 84 is a diagram showing the configuration of encoded data and the method of storing encoded data in a NAL unit according to embodiment 7. FIG. 85 is a flowchart showing an example of processing regarding the initialization of CABAC in the decoding of position information or attribute information according to Embodiment 7. FIG. 86 is a flowchart of the encoding process of point cloud data according to Embodiment 7. FIG. 87 is a flowchart illustrating an example of a process for updating additional information according to Embodiment 7. FIG. 88 is a flowchart illustrating an example of a CABAC initialization process according to Embodiment 7. FIG. 89 is a flowchart of the decoding process of point cloud data according to embodiment 7. FIG. 90 is a flowchart showing an example of a process for initializing a CABAC decoding unit according to embodiment 7. FIG. 91 is a drawing showing an example of a tile and a slice according to embodiment 7. FIG. 92 is a flowchart illustrating an example of a method for initializing CABAC and determining context initial values ​​according to embodiment 7. FIG. 93 is a diagram showing an example of a map viewed from the top surface being divided into tiles using point cloud data obtained from LiDAR according to Embodiment 7. FIG. 94 is a flowchart illustrating another example of a method for initializing CABAC and determining context initial values ​​according to embodiment 7. FIG. 95 is a diagram for explaining the processing of the quantization unit and the inverse quantization unit according to embodiment 8. FIG. 96 is a diagram illustrating the default value of the quantization value and the quantization delta according to embodiment 8. FIG. 97 is a block diagram showing the configuration of a first encoding unit included in a three-dimensional data encoding device according to embodiment 8. FIG. 98 is a block diagram showing the configuration of a divided section according to embodiment 8. FIG. 99 is a block diagram showing the configuration of a position information encoding unit and an attribute information encoding unit according to embodiment 8. FIG. 100 is a block diagram showing the configuration of the first decoding unit according to embodiment 8. FIG. 101 is a block diagram showing the configuration of a position information decoding unit and an attribute information decoding unit according to embodiment 8. FIG. 102 is a flowchart illustrating an example of processing for determining quantization values ​​in encoding position information or encoding attribute information according to embodiment 8. FIG. 103 is a flowchart illustrating an example of decoding processing of location information and attribute information according to embodiment 8. FIG. 104 is a drawing for explaining a first example of a method for transmitting quantization parameters according to embodiment 8. FIG. 105 is a drawing for explaining a second example of a method for transmitting quantization parameters according to embodiment 8. FIG. 106 is a drawing for explaining a third example of a method for transmitting quantization parameters according to embodiment 8. FIG. 107 is a flowchart of the encoding process of point cloud data according to embodiment 8. FIG. 108 is a flowchart showing an example of a process for determining a QP value and updating additional information according to embodiment 8. FIG. 109 is a flowchart illustrating an example of a process for encoding a determined QP value according to embodiment 8. FIG. 110 is a flowchart of decoding processing of point cloud data according to embodiment 8. FIG. 111 is a flowchart illustrating an example of a process for acquiring a QP value according to embodiment 8 and decoding the QP value of a slice or tile. FIG. 112 is a drawing showing an example of the syntax of a GPS according to embodiment 8. FIG. 113 is a drawing showing an example of the syntax of an APS according to embodiment 8. FIG. 114 is a drawing showing an example of the syntax of a header of position information according to embodiment 8. FIG. 115 is a drawing showing an example of the syntax of a header of attribute information according to embodiment 8. FIG. 116 is a drawing for explaining another example of a method for transmitting quantization parameters according to embodiment 8. FIG. 117 is a drawing for explaining another example of a method for transmitting quantization parameters according to embodiment 8. FIG. 118 is a drawing for explaining a ninth example of a method for transmitting quantization parameters according to embodiment 8. FIG. 119 is a drawing for explaining an example of controlling the QP value according to embodiment 8. FIG. 120 is a flowchart illustrating an example of a method for determining a QP value based on the quality of an object according to embodiment 8. FIG. 121 is a flowchart showing an example of a method for determining a QP value based on rate control according to embodiment 8. FIG. 122 is a flowchart of encoding processing according to embodiment 8. FIG. 123 is a flowchart of decoding processing according to embodiment 8. FIG. 124 is a drawing for explaining an example of a method for transmitting quantization parameters according to embodiment 9. FIG. 125 is a drawing showing a first example of the syntax of an APS according to embodiment 9, and the syntax of a header of attribute information. FIG. 126 is a drawing showing a second example of the syntax of an APS according to embodiment 9. FIG. 127 is a drawing showing a second example of the syntax of a header of attribute information according to embodiment 9. FIG. 128 is a diagram showing the relationship between the headers of SPS, APS, and attribute information according to embodiment 9. FIG. 129 is a flowchart of encoding processing according to embodiment 9. FIG. 130 is a flowchart of decoding processing according to embodiment 9. Specific details for implementing the invention

[0013] A three-dimensional data encoding method according to one aspect of the present disclosure encodes a plurality of attribute information having each of a plurality of three-dimensional points using parameters, and generates a bit stream comprising the encoded plurality of attribute information, control information, and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a different type of attribute information, and the plurality of first attribute control information each corresponds to the plurality of attribute information and each of the plurality of first attribute control information includes a first identification information indicating that it corresponds to any one of the plurality of type information.

[0014] According to this, since the first attribute control information generates a bit stream including first identification information for specifying the type of attribute information corresponding to the first attribute control information, the three-dimensional data decoding device that receives the bit stream can correctly and efficiently decode the attribute information of the three-dimensional point.

[0015] For example, the plurality of type information is stored in a predetermined order in the control information, and the first identification information may indicate that the first attribute control information including the first identification information corresponds to the type information of one of the sequence numbers in the predetermined order.

[0016] According to this, since type information appears in a predetermined order without adding information indicating type information, the amount of data in the bit stream can be reduced and the amount of transmission of the bit stream can be reduced.

[0017] For example, the bit stream may also include a plurality of second attribute control information corresponding to the plurality of attribute information, and each of the plurality of second attribute control information may include a reference value of a parameter used for encoding the corresponding attribute information.

[0018] According to this, since each of the plurality of second attribute control information includes a reference value of a parameter, the attribute information corresponding to said second attribute control information can be encoded using the reference value.

[0019] For example, the first attribute control information may include difference information which is the difference from the reference value of the parameter.

[0020] For this reason, encoding efficiency can be improved.

[0021] For example, the bit stream may also include a plurality of second attribute control information corresponding to the plurality of attribute information, and each of the plurality of second attribute control information may have second identification information indicating that it corresponds to any one of the plurality of type information.

[0022] According to this, since a bit stream including second identification information for specifying the type of attribute information corresponding to the second attribute control information is generated, a bit stream capable of correctly and efficiently decoding the attribute information of a three-dimensional point can be generated.

[0023] For example, each of the plurality of first attribute control information has N fields in which N parameters (N is 2 or more) are stored, and in a specific first attribute control information corresponding to a specific type of attribute among the plurality of first attribute control information, one of the N fields may include a value indicating that it is invalid.

[0024] For this reason, the three-dimensional data decoding device that receives the bit stream can correctly and efficiently decode the attribute information of the three-dimensional point by using the first identification information to specify the type of the first attribute information and omitting the decoding process in the case of the specific first attribute control information.

[0025] For example, in the above encoding, the plurality of attribute information may be quantized using a quantization parameter as the parameter.

[0026] According to this, since the parameters are represented using the difference from the reference value, the encoding efficiency resulting from quantization can be improved.

[0027] In addition, a three-dimensional data decoding method according to one aspect of the present disclosure comprises acquiring a plurality of encoded attribute information and a parameter by acquiring a bit stream and decoding the encoded plurality of attribute information using the parameter to decode the plurality of attribute information having each of the plurality of three-dimensional points, wherein the bit stream includes control information and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a different type of attribute information, wherein the plurality of first attribute control information each corresponds to the plurality of attribute information and each of the plurality of first attribute control information includes a first identification information indicating that it corresponds to any one of the plurality of type information.

[0028] According to this, since the type of attribute information corresponding to the first attribute control information can be specified using the first identification information, the attribute information of the three-dimensional point can be decoded correctly and efficiently.

[0029] For example, the plurality of type information is stored in a predetermined order in the control information, and the first identification information may indicate that the first attribute control information including the first identification information corresponds to the type information of one of the sequence numbers in the predetermined order.

[0030] According to this, since type information appears in a predetermined order without adding information indicating type information, the amount of data in the bit stream can be reduced and the amount of transmission of the bit stream can be reduced.

[0031] For example, the bit stream may also include a plurality of second attribute control information corresponding to the plurality of attribute information, and each of the plurality of second attribute control information may include a reference value of a parameter used for encoding the corresponding attribute information.

[0032] According to this, since the second attribute control information can decode the corresponding attribute information using a reference value, the attribute information of the three-dimensional point can be decoded correctly and efficiently.

[0033] For example, the first attribute control information may include difference information which is the difference from the reference value of the parameter.

[0034] According to this, since attribute information can be decoded using reference values ​​and difference information, attribute information of three-dimensional points can be decoded correctly and efficiently.

[0035] For example, the bit stream may also include a plurality of second attribute control information corresponding to the plurality of attribute information, and each of the plurality of second attribute control information may have second identification information indicating that it corresponds to any one of the plurality of type information.

[0036] According to this, since the type of attribute information corresponding to the second attribute control information can be specified using the second identification information, the attribute information of the three-dimensional point can be decoded correctly and efficiently.

[0037] For example, each of the plurality of first attribute control information has a plurality of fields in which a plurality of parameters are stored, and in the decoding, a parameter stored in a specific field among the plurality of fields of a specific first attribute control information corresponding to a specific type of attribute among the plurality of first attribute control information may be ignored.

[0038] According to this, by using first identification information to specify the type of first attribute information and omitting decoding processing in the case of specific first attribute control information, the attribute information of a three-dimensional point can be decoded correctly and efficiently.

[0039] For example, in the above decoding, the encoded plurality of attribute information may be inversely quantized using the quantization parameter as the parameter.

[0040] According to this, the attribute information of a three-dimensional point can be correctly decoded.

[0041] In addition, a three-dimensional data encoding device according to one aspect of the present disclosure comprises a processor and a memory, wherein the processor uses the memory to encode a plurality of attribute information having each of a plurality of three-dimensional points using parameters, and generates a bit stream including the encoded plurality of attribute information, control information, and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a different type of attribute information, and the plurality of first attribute control information includes (i) each of the plurality of attribute information and (ii) first identification information indicating that it corresponds to any one of the plurality of type information.

[0042] According to this, since a bit stream including first identification information for specifying the type of attribute information corresponding to the first attribute control information is generated, a bit stream capable of correctly and efficiently decoding the attribute information of a three-dimensional point can be generated.

[0043] In addition, a three-dimensional data decoding device according to one aspect of the present disclosure comprises a processor and a memory, wherein the processor acquires a plurality of encoded attribute information and a parameter by acquiring a bit stream using the memory, and decodes a plurality of attribute information having each of a plurality of three-dimensional points by decoding the encoded plurality of attribute information using the parameter, wherein the bit stream includes control information and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a type of attribute information different from each other, and wherein the plurality of first attribute control information includes (i) each corresponding to the plurality of attribute information and (ii) a first identification information indicating that it corresponds to any one of the plurality of type information.

[0044] According to this, since the type of attribute information corresponding to the first attribute control information can be specified using the first identification information, the attribute information of the three-dimensional point can be decoded correctly and efficiently.

[0045] In addition, these comprehensive or specific embodiments may be realized as a system, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, and recording medium.

[0046] Hereinafter, embodiments will be described in detail with reference to the drawings. Furthermore, all embodiments described below represent specific embodiments of the present disclosure. The numerical values, shapes, materials, components, arrangement positions and connection types of components, steps, and sequence of steps shown in the following embodiments are examples and are not well known facts that limit the present disclosure. Additionally, regarding the components in the following embodiments, any components not described in the independent claim representing the highest concept are described as arbitrary components.

[0047] (Embodiment 1)

[0048] When using encoded point cloud data in actual devices or services, it is desirable to transmit and receive necessary information according to the application to conserve network bandwidth. However, until now, such a function has not existed in the encoding structure of three-dimensional data, nor has there been an encoding method for it.

[0049] In this embodiment, a three-dimensional data encoding method and a three-dimensional data encoding device for providing a function to transmit and receive information necessary according to the application in three-dimensional point cloud encoded data, a three-dimensional data decoding method and a three-dimensional data decoding device for decoding said encoded data, a three-dimensional data multiplexing method for multiplexing said encoded data, and a three-dimensional data transmission method for transmitting said encoded data are described.

[0050] In particular, currently, a first encoding method and a second encoding method are being considered as encoding methods (encoding schemes) for point cloud data, but the composition of the encoded data and the method for storing the encoded data in a system format are not defined, and there is a problem that MUX processing (multiplexing) in the encoding unit, or transmission or storage, cannot be performed as is.

[0051] Also, there is no method to support a format in which two codecs, such as the first encoding method and the second encoding method, are mixed, such as PCC (Point Cloud Compression).

[0052] In this embodiment, the configuration of PCC encoded data in which two codecs of the first encoding method and the second encoding method are mixed, and a method for storing the encoded data in a system format are described.

[0053] First, the configuration of a three-dimensional data (point cloud data) encoding and decoding system according to the present embodiment will be explained. FIG. 1 is a diagram showing an example of the configuration of a three-dimensional data encoding and decoding system according to the present embodiment. As shown in FIG. 1, the three-dimensional data encoding and decoding system includes a three-dimensional data encoding system (4601), a three-dimensional data decoding system (4602), a sensor terminal (4603), and an external connection unit (4604).

[0054] The three-dimensional data encoding system (4601) generates encoded data or multiplexed data by encoding point cloud data, which is three-dimensional data. Additionally, the three-dimensional data encoding system (4601) may be a three-dimensional data encoding device realized by a single device, or a system realized by multiple devices. Furthermore, the three-dimensional data encoding device may include some of the multiple processing units included in the three-dimensional data encoding system (4601).

[0055] The three-dimensional data encoding system (4601) includes a point cloud data generation system (4611), a presentation unit (4612), an encoding unit (4613), a multiplexing unit (4614), an input / output unit (4615), and a control unit (4616). The point cloud data generation system (4611) includes a sensor information acquisition unit (4617) and a point cloud data generation unit (4618).

[0056] The sensor information acquisition unit (4617) acquires sensor information from the sensor terminal (4603) and outputs the sensor information to the point cloud data generation unit (4618). The point cloud data generation unit (4618) generates point cloud data from the sensor information and outputs the point cloud data to the encoding unit (4613).

[0057] The presentation unit (4612) presents sensor information or point cloud data to the user. For example, the presentation unit (4612) displays information or an image based on the sensor information or point cloud data.

[0058] The encoding unit (4613) encodes (compresses) point cloud data and outputs the obtained encoded data, control information obtained during the encoding process, and other additional information to the multiplexing unit (4614). The additional information includes, for example, sensor information.

[0059] The multiplexing unit (4614) generates multiplexed data by multiplexing the encoded data input from the encoding unit (4613), control information, and additional information. The format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.

[0060] The input / output unit (4615) (e.g., a communication unit or an interface) outputs multiplexed data to the outside. Alternatively, the multiplexed data is stored in an accumulation unit such as an internal memory. The control unit (4616) (or application execution unit) controls each processing unit. That is, the control unit (4616) performs control such as encoding and multiplexing.

[0061] Additionally, sensor information may be input to the encoding unit (4613) or the multiplexing unit (4614). Also, the input / output unit (4615) may output point cloud data or encoded data to the outside as is.

[0062] The transmission signal (multiplexed data) output from the three-dimensional data encoding system (4601) is input to the three-dimensional data decoding system (4602) through the external connection unit (4604).

[0063] The three-dimensional data decoding system (4602) generates point cloud data, which is three-dimensional data, by decoding encoded data or multiplexed data. Additionally, the three-dimensional data decoding system (4602) may be a three-dimensional data decoding device realized by a single device, or a system realized by multiple devices. Furthermore, the three-dimensional data decoding device may include some of the multiple processing units included in the three-dimensional data decoding system (4602).

[0064] The three-dimensional data decoding system (4602) includes a sensor information acquisition unit (4621), an input / output unit (4622), a demultiplexer unit (4623), a decoding unit (4624), a presentation unit (4625), a user interface (4626), and a control unit (4627).

[0065] The sensor information acquisition unit (4621) acquires sensor information from the sensor terminal (4603).

[0066] The input / output unit (4622) acquires a transmission signal, decodes multiplexed data (file format or packet) from the transmission signal, and outputs the multiplexed data to the demultiplexer (4623).

[0067] The demultiplexer (4623) obtains encoded data, control information, and additional information from the multiplexed data, and outputs the encoded data, control information, and additional information to the decoder (4624).

[0068] The decoding unit (4624) reconstructs point cloud data by decoding the encoded data.

[0069] The presentation unit (4625) presents point cloud data to the user. For example, the presentation unit (4625) displays information or images based on the point cloud data. The user interface (4626) obtains instructions based on the user's operation. The control unit (4627) (or application execution unit) controls each processing unit. That is, the control unit (4627) performs control such as demultiplexing, decoding, and presentation.

[0070] Additionally, the input / output unit (4622) may acquire point cloud data or encoded data directly from the outside. Also, the presentation unit (4625) may acquire additional information such as sensor information and present information based on the additional information. Also, the presentation unit (4625) may perform presentation based on user instructions acquired from the user interface (4626).

[0071] The sensor terminal (4603) generates sensor information, which is information obtained from the sensor. The sensor terminal (4603) is a terminal equipped with a sensor or a camera, and, for example, may be a moving object such as a car, a flying object such as an airplane, a portable terminal, or a camera.

[0072] Sensor information obtainable from the sensor terminal (4603) is, for example, (1) the distance between the sensor terminal (4603) and an object, or the reflectance of the object, obtained from a LIDAR, millimeter-wave radar, or infrared sensor, (2) the distance between the camera and an object or the reflectance of the object, obtained from multiple monocular camera images or stereo camera images. In addition, the sensor information may include the attitude, direction, gyroscope (angular velocity), position (GPS information or altitude), speed, or acceleration of the sensor. In addition, the sensor information may include temperature, atmospheric pressure, humidity, or magnetic field.

[0073] The external connection part (4604) is realized by an integrated circuit (LSI or IC), an external storage part, communication with a cloud server via the internet, or broadcasting, etc.

[0074] Next, point cloud data will be described. Figure 2 is a diagram showing the structure of point cloud data. Figure 3 is a diagram showing an example of the structure of a data file in which information about point cloud data is described.

[0075] Point cloud data contains data from multiple points. The data for each point includes location information (three-dimensional coordinates) and attribute information regarding that location information. A collection of these points is called a point cloud. For example, a point cloud represents the three-dimensional shape of an object.

[0076] Position information, such as three-dimensional coordinates, is sometimes referred to as geometry. Additionally, the data for each point may include attribute information of multiple attribute types. Attribute types include, for example, color or reflectance.

[0077] One attribute information may be mapped to one location information, or one attribute information having multiple different attribute types may be mapped to one location information. Also, multiple attribute information of the same attribute type may be mapped to one location information.

[0078] The example of the data file configuration shown in Figure 3 is an example where location information and attribute information correspond one-to-one, and shows the location information and attribute information of N points constituting the point cloud data.

[0079] Position information is, for example, information on the three axes x, y, and z. Attribute information is, for example, color information in RGB. Representative data files include ply files.

[0080] Next, the types of point cloud data will be explained. Figure 4 is a diagram showing the types of point cloud data. As shown in Figure 4, point cloud data includes static objects and dynamic objects.

[0081] A static object is three-dimensional point cloud data at an arbitrary time (a specific point in time). A dynamic object is three-dimensional point cloud data that changes over time. From now on, the three-dimensional point cloud data at a specific time is referred to as a PCC frame, or simply a frame.

[0082] The object may be a point cloud with a limited area, like conventional image data, or a large-scale point cloud with an unrestricted area, like map information.

[0083] In addition, point cloud data of various densities may exist, and sparse and dense point cloud data may also be present.

[0084] Below, details of each processing unit will be described. Sensor information is acquired in various ways, such as distance sensors like LIDAR or rangefinders, stereo cameras, or combinations of multiple monocular cameras. The point cloud data generation unit (4618) generates point cloud data based on the sensor information obtained from the sensor information acquisition unit (4617). The point cloud data generation unit (4618) generates location information as point cloud data and adds attribute information regarding the location information to the location information.

[0085] The point cloud data generation unit (4618) may process the point cloud data when generating location information or adding attribute information. For example, the point cloud data generation unit (4618) may reduce the amount of data by deleting point clouds with overlapping locations. In addition, the point cloud data generation unit (4618) may transform the location information (position shift, rotation, or normalization, etc.) and render the attribute information.

[0086] In addition, in FIG. 1, the point cloud data generation system (4611) is included in the three-dimensional data encoding system (4601), but it may be installed independently outside the three-dimensional data encoding system (4601).

[0087] The encoding unit (4613) generates encoded data by encoding point cloud data according to a predetermined encoding method. There are two main types of encoding methods. The first is an encoding method using position information, and this encoding method is hereinafter referred to as the first encoding method. The second is an encoding method using a video codec, and this encoding method is hereinafter referred to as the second encoding method.

[0088] The decoding unit (4624) decodes point cloud data by decoding the encoded data according to a predetermined encoding method.

[0089] The multiplexer (4614) generates multiplexed data by multiplexing encoded data using an existing multiplexing method. The generated multiplexed data is transmitted or accumulated. In addition to PCC encoded data, the multiplexer (4614) multiplexes other media such as video, audio, subtitles, applications, files, or reference time information. Furthermore, the multiplexer (4614) may also multiplex attribute information related to sensor information or point cloud data.

[0090] Multiplexing methods or file formats include ISOBMFF, ISOBMFF-based transmission methods such as MPEG-DASH, MMT, MPEG-2 TS Systems, and RMP.

[0091] The demultiplexer (4623) extracts PCC encoded data, other media, and visual information, etc. from the multiplexed data.

[0092] The input / output unit (4615) transmits multiplexed data using a method suited to a transmitting medium or a storing medium, such as broadcasting or communication. The input / output unit (4615) may communicate with another device via the Internet or with a storage unit such as a cloud server.

[0093] As communication protocols, http, ftp, TCP, or UDP are used. PULL communication methods may be used, or PUSH communication methods may be used.

[0094] Either wired or wireless transmission may be used. For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), or coaxial cable, etc., are used. For wireless transmission, wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), or millimeter wave, etc., are used.

[0095] In addition, broadcasting systems such as DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 are used.

[0096] FIG. 5 is a diagram showing the configuration of a first encoding unit (4630), which is an example of an encoding unit (4613) that performs encoding of the first encoding method. FIG. 6 is a block diagram of the first encoding unit (4630). The first encoding unit (4630) generates encoded data (encoded stream) by encoding point cloud data using the first encoding method. This first encoding unit (4630) includes a position information encoding unit (4631), an attribute information encoding unit (4632), an additional information encoding unit (4633), and a multiplexing unit (4634).

[0097] The first encoding unit (4630) has the characteristic of performing encoding while being conscious of the three-dimensional structure. In addition, the first encoding unit (4630) has the characteristic that the attribute information encoding unit (4632) performs encoding using information obtained from the position information encoding unit (4631). The first encoding method is also called GPCC (Geometry based PCC).

[0098] Point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and includes position information, attribute information, and other additional information (metadata). Position information is input to the position information encoding unit (4631), attribute information is input to the attribute information encoding unit (4632), and additional information is input to the additional information encoding unit (4633).

[0099] The location information encoding unit (4631) generates encoded location information (Compressed Geometry), which is encoded data, by encoding location information. For example, the location information encoding unit (4631) encodes location information using an N-ary tree structure such as an octal tree. Specifically, in an octal tree, the target space is divided into 8 nodes (subspaces), and 8-bit information (occupancy code) indicating whether a point cloud is included in each node is generated. Also, the node containing the point cloud is further divided into 8 nodes, and 8-bit information indicating whether a point cloud is included in each of the 8 nodes is generated. This process is repeated until the number of point clouds included in a predetermined hierarchy or node is below a threshold.

[0100] The attribute information encoding unit (4632) generates encoded attribute information (Compressed Attribute), which is encoded data, by encoding using the configuration information generated by the location information encoding unit (4631). For example, the attribute information encoding unit (4632) determines a reference point (reference node) to be referenced in encoding the target point (target node) of the processing target based on the octal tree structure generated by the location information encoding unit (4631). For example, the attribute information encoding unit (4632) references a node among the surrounding nodes or adjacent nodes where the parent node in the octal tree is the same as the target node. Furthermore, the method of determining the reference relationship is not limited to this.

[0101] Additionally, the encoding process of attribute information may include at least one of quantization processing, prediction processing, and arithmetic encoding processing. In this case, "reference" refers to using a reference node to calculate the predicted value of the attribute information, or using the state of a reference node (e.g., occupancy information indicating whether the reference node contains a point cloud) to determine the encoding parameters. For example, the encoding parameters are quantization parameters in the quantization processing, or context in arithmetic encoding, etc.

[0102] The additional information encoding unit (4633) generates encoded additional information (Compressed MetaData), which is encoded data, by encoding compressible data among the additional information.

[0103] The multiplexer (4634) generates a compressed stream, which is encoded data, by multiplexing the encoding location information, encoding attribute information, encoding additional information, and other additional information. The generated compressed stream is output to a processing unit of a system layer not shown.

[0104] Next, a first decoding unit (4640), which is an example of a decoding unit (4624) that performs decoding of the first encoding method, will be described. FIG. 7 is a diagram showing the configuration of the first decoding unit (4640). FIG. 8 is a block diagram of the first decoding unit (4640). The first decoding unit (4640) generates point cloud data by decoding encoded data (encoded stream) encoded by the first encoding method using the first encoding method. This first decoding unit (4640) includes a demultiplexer (4641), a position information decoding unit (4642), an attribute information decoding unit (4643), and an additional information decoding unit (4644).

[0105] A compressed stream, which is encoded data, is input to the first decoding unit (4640) from the processing unit of the system layer that is not shown in the city.

[0106] The demultiplexer (4641) separates the compressed geometry, compressed attribute, compressed metadata, and other additional information from the compressed data.

[0107] The position information decoding unit (4642) generates position information by decoding encoded position information. For example, the position information decoding unit (4642) restores position information of a point group represented in three-dimensional coordinates from encoded position information represented in an N-ary tree structure such as an octal tree.

[0108] The attribute information decoding unit (4643) decodes encoded attribute information based on the configuration information generated by the location information decoding unit (4642). For example, the attribute information decoding unit (4643) determines a reference point (reference node) to be referenced in decoding a target point (target node) of a processing target based on the octal tree structure obtained from the location information decoding unit (4642). For example, the attribute information decoding unit (4643) references a node among surrounding nodes or adjacent nodes where the parent node in the octal tree is the same as the target node. Furthermore, the method for determining the reference relationship is not limited to this.

[0109] Additionally, the decoding process of attribute information may include at least one of inverse quantization processing, prediction processing, and arithmetic decoding processing. In this case, "reference" refers to using a reference node to calculate the predicted value of the attribute information, or using the state of a reference node (e.g., occupancy information indicating whether the reference node contains a point cloud) to determine decoding parameters. For example, decoding parameters are quantization parameters in the inverse quantization processing, or context in arithmetic decoding, etc.

[0110] The additional information decoding unit (4644) generates additional information by decoding the encoded additional information. Additionally, the first decoding unit (4640) uses the additional information required for decoding the location information and attribute information during decoding, and outputs the additional information required for the application to the outside.

[0111] Next, a second encoding unit (4650), which is an example of an encoding unit (4613) that performs encoding of the second encoding method, will be described. FIG. 9 is a diagram showing the configuration of the second encoding unit (4650). FIG. 10 is a block diagram of the second encoding unit (4650).

[0112] The second encoding unit (4650) generates encoded data (encoded stream) by encoding point cloud data using a second encoding method. This second encoding unit (4650) includes an additional information generating unit (4651), a position image generating unit (4652), an attribute image generating unit (4653), an image encoding unit (4654), an additional information encoding unit (4655), and a multiplexing unit (4656).

[0113] The second encoding unit (4650) is characterized by generating a position image and an attribute image by projecting a three-dimensional structure onto a two-dimensional image, and encoding the generated position image and attribute image using a conventional image encoding method. The second encoding method is also called VPCC (Video based PCC).

[0114] Point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and includes position information, attribute information, and other additional information (metadata).

[0115] The additional information generation unit (4651) generates map information of multiple two-dimensional images by projecting a three-dimensional structure onto a two-dimensional image.

[0116] The position image generation unit (4652) generates a position image (Geometry Image) based on position information and map information generated by the additional information generation unit (4651). This position image is, for example, a distance image in which depth appears as a pixel value. In addition, this distance image may be an image in which multiple point groups are viewed from a single viewpoint (an image in which multiple point groups are projected onto a single two-dimensional plane), multiple images in which multiple point groups are viewed from multiple viewpoints, or a single image in which these multiple images are integrated.

[0117] The attribute image generation unit (4653) generates an attribute image based on attribute information and map information generated by the additional information generation unit (4651). This attribute image is, for example, an image in which attribute information (e.g., color (RGB)) appears as pixel values. In addition, this image may be an image in which multiple point groups are viewed from a single viewpoint (an image in which multiple point groups are projected onto a single two-dimensional plane), multiple images in which multiple point groups are viewed from multiple viewpoints, or a single image in which these multiple images are integrated.

[0118] The image encoding unit (4654) generates encoded data, namely a compressed geometry image and a compressed attribute image, by encoding the position image and the attribute image using an image encoding method. Additionally, any known encoding method may be used as the image encoding method. For example, the image encoding method may be AVC or HEVC.

[0119] The additional information encoding unit (4655) generates compressed additional information by encoding additional information and map information included in the point cloud data.

[0120] The multiplexer (4656) generates a compressed stream, which is encoded data, by multiplexing an encoded position image, an encoded attribute image, encoded additional information, and other additional information. The generated compressed stream is output to a processing unit of a system layer not shown.

[0121] Next, a second decoding unit (4660), which is an example of a decoding unit (4624) that performs decoding of the second encoding method, will be described. FIG. 11 is a diagram showing the configuration of the second decoding unit (4660). FIG. 12 is a block diagram of the second decoding unit (4660). The second decoding unit (4660) generates point cloud data by decoding encoded data (encoded stream) encoded by the second encoding method using the second encoding method. This second decoding unit (4660) includes a demultiplexer (4661), an image decoding unit (4662), an additional information decoding unit (4663), a location information generation unit (4664), and an attribute information generation unit (4665).

[0122] A compressed stream, which is encoded data, is input to the second decoding unit (4660) from the processing unit of the system layer that is not shown in the city.

[0123] The demultiplexer (4661) separates the compressed geometry image, the compressed attribute image, the compressed metadata, and other additional information from the encoded data.

[0124] The image decoder (4662) generates a position image and an attribute image by decoding the encoded position image and the encoded attribute image using an image encoding method. Additionally, any known encoding method may be used as the image encoding method. For example, the image encoding method may be AVC or HEVC.

[0125] The additional information decoding unit (4663) generates additional information including map information, etc. by decoding the encoded additional information.

[0126] The location information generation unit (4664) generates location information using location images and map information. The attribute information generation unit (4665) generates attribute information using attribute images and map information.

[0127] The second decoding unit (4660) uses additional information required for decoding during decoding and outputs additional information required for the application to the outside.

[0128] The following describes the challenges associated with the PCC encoding method. FIG. 13 is a diagram showing a protocol stack related to PCC encoded data. FIG. 13 illustrates an example of multiplexing data from other media, such as video (e.g., HEVC) or audio, with PCC encoded data, and transmitting or storing it.

[0129] Multiplexing schemes and file formats have the function of multiplexing various encoded data and transmitting or storing them. To transmit or store encoded data, the data must be converted into the format of the multiplexing scheme. For example, HEVC specifies a technique for storing encoded data in a data structure called a NAL unit and storing the NAL unit in an ISOBMFF.

[0130] Meanwhile, although a first encoding method (Codec1) and a second encoding method (Codec2) are currently being considered as encoding methods for point cloud data, the composition of the encoded data and the method for storing the encoded data in a system format are not defined, and there is a problem that MUX processing (multiplexing), transmission, and storage in the encoding unit cannot be performed as they are.

[0131] In addition, if no specific encoding method is described thereafter, it shall be considered as either the first encoding method or the second encoding method.

[0132] Hereinafter, a method for defining a NAL unit according to the present embodiment will be described. For example, in conventional codecs such as HEVC, a single format NAL unit is defined for a single codec. However, there has not been a method to support a format in which two codecs (hereinafter referred to as PCC codecs) of a first encoding method and a second encoding method are mixed, such as PCC.

[0133] First, an encoding unit (4670) having the functions of both the first encoding unit (4630) and the second encoding unit (4650) described above, and a decoding unit (4680) having the functions of both the first decoding unit (4640) and the second decoding unit (4660) will be described.

[0134] FIG. 14 is a block diagram of an encoding unit (4670) according to the present embodiment. The encoding unit (4670) includes the first encoding unit (4630) and the second encoding unit (4650) described above, and a multiplexing unit (4671). The multiplexing unit (4671) multiplexes the encoded data generated by the first encoding unit (4630) and the encoded data generated by the second encoding unit (4650), and outputs the obtained encoded data.

[0135] FIG. 15 is a block diagram of a decoding unit (4680) according to the present embodiment. This decoding unit (4680) includes the first decoding unit (4640) and the second decoding unit (4660) described above, and a demultiplexer (4681). The demultiplexer (4681) extracts encoded data in which the first encoding method is used and encoded data in which the second encoding method is used from the input encoded data. The demultiplexer (4681) outputs the encoded data in which the first encoding method is used to the first decoding unit (4640) and outputs the encoded data in which the second encoding method is used to the second decoding unit (4660).

[0136] With the above configuration, the encoding unit (4670) can encode point cloud data by selectively using the first encoding method and the second encoding method. Additionally, the decoding unit (4680) can decode encoded data encoded using the first encoding method, encoded data encoded using the second encoding method, and encoded data encoded using both the first encoding method and the second encoding method.

[0137] For example, the encoding unit (4670) may switch the encoding method (first encoding method and second encoding method) in units of point cloud data or frames. Also, the encoding unit (4670) may switch the encoding method in units that can be encoded.

[0138] The encoding unit (4670) generates encoding data (encoding stream) including identification information of a PCC codec, for example.

[0139] The demultiplexer (4681) included in the decoding unit (4680) identifies data using, for example, identification information of a PCC codec. The demultiplexer (4681) outputs the data to the first decoding unit (4640) if the data is data encoded by the first encoding method, and outputs the data to the second decoding unit (4660) if the data is data encoded by the second encoding method.

[0140] In addition, the encoding unit (4670) may transmit information indicating whether both encoding methods were used or only one encoding method was used, in addition to the identification information of the PCC codec, as control information.

[0141] Next, the encoding process according to the present embodiment will be described. FIG. 16 is a flowchart of the encoding process according to the present embodiment. By using identification information of the PCC codec, encoding processing corresponding to multiple codecs becomes possible.

[0142] First, the encoding unit (4670) encodes PCC data using either or both of the first encoding method and the second encoding method codecs (S4681).

[0143] If the codec used is a second encoding method (second encoding method in S4682), the encoding unit (4670) sets the pcc_codec_type included in the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4683). Next, the encoding unit (4670) sets an identifier for the NAL unit for the second encoding method in the pcc_nal_unit_type of the NAL unit header (S4684). Then, the encoding unit (4670) creates a NAL unit containing encoded data in the payload using the set NAL unit header. Then, the encoding unit (4670) transmits the created NAL unit (S4685).

[0144] Meanwhile, if the codec used is the first encoding method (the first encoding method in S4682), the encoding unit (4670) sets the pcc_codec_type included in the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded by the first encoding method (S4686). Next, the encoding unit (4670) sets the identifier of the NAL unit for the first encoding method in the pcc_nal_unit_type included in the NAL unit header (S4687). Next, the encoding unit (4670) has the set NAL unit header and generates a NAL unit containing encoded data in the payload. Then, the encoding unit (4670) transmits the generated NAL unit (S4685).

[0145] Next, decoding processing according to the present embodiment will be described. FIG. 17 is a flowchart of decoding processing according to the present embodiment. By using identification information of the PCC codec, decoding processing corresponding to multiple codecs becomes possible.

[0146] First, the decoding unit (4680) receives a NAL unit (S4691). For example, this NAL unit is generated by processing in the encoding unit (4670) described above.

[0147] Next, the decoding unit (4680) determines whether the pcc_codec_type included in the NAL unit header represents a first encoding method or a second encoding method (S4692).

[0148] If pcc_codec_type indicates a second encoding method (second encoding method in S4692), the decoding unit (4680) determines that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4693). Then, the second decoding unit (4660) identifies the data by determining that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the second encoding method (S4694). Then, the decoding unit (4680) decodes the PCC data using the decoding process of the second encoding method (S4695).

[0149] Meanwhile, if pcc_codec_type represents the first encoding method (the first encoding method in S4692), the decoding unit (4680) determines that the data included in the payload of the NAL unit is data encoded by the first encoding method (S4696). Then, the decoding unit (4680) identifies the data by determining that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first encoding method (S4697). Then, the decoding unit (4680) decodes the PCC data using the decoding process of the first encoding method (S4698).

[0150] As described above, a three-dimensional data encoding device according to one aspect of the present disclosure generates an encoding stream by encoding three-dimensional data (e.g., point cloud data), and stores in control information (e.g., a parameter set) of the encoding stream information indicating the encoding method used for the encoding among a first encoding method and a second encoding method (e.g., identification information of a codec).

[0151] According to this, when a three-dimensional data decoder decodes a encoded stream generated by the said three-dimensional data encoding device, it can determine the encoding method used for encoding by using information stored in the control information. Therefore, the three-dimensional data decoder can correctly decode the encoded stream even when multiple encoding methods are used.

[0152] For example, the above three-dimensional data includes position information. In the encoding, the three-dimensional data encoding device encodes the position information. In the storage, the three-dimensional data encoding device stores information indicating the encoding method used for encoding the position information among the first encoding method and the second encoding method in the control information of the position information.

[0153] For example, the above three-dimensional data includes position information and attribute information. In the encoding, the three-dimensional data encoding device encodes the position information and the attribute information. In the storage, the three-dimensional data encoding device stores information indicating the encoding method used for encoding the position information among the first encoding method and the second encoding method in the control information of the position information, and stores information indicating the encoding method used for encoding the attribute information among the first encoding method and the second encoding method in the control information of the attribute information.

[0154] According to this, since different encoding methods can be used for location information and attribute information, encoding efficiency can be improved.

[0155] For example, the three-dimensional data encoding method also stores the encoding stream in one or more units (e.g., NAL units).

[0156] For example, the unit has a format common to the first encoding method and the second encoding method, and includes information that has a definition independent of the first encoding method and the second encoding method (e.g., pcc_nal_unit_type) as information indicating the type of data included in the unit.

[0157] For example, the unit has a format independent of the first encoding method and the second encoding method, and includes information indicating the type of data included in the unit, which has a definition independent of the first encoding method and the second encoding method (e.g., codec1_nal_unit_type or codec2_nal_unit_type).

[0158] For example, the unit has a format common to the first encoding method and the second encoding method, and includes information having a definition common to the first encoding method and the second encoding method (e.g., pcc_nal_unit_type) as information indicating the type of data included in the unit.

[0159] For example, a three-dimensional data encoding device is equipped with a processor and memory, and the processor uses the memory to perform the above processing.

[0160] In addition, the three-dimensional data decoding device according to the present embodiment determines the encoding method used for encoding the encoding stream based on information indicating the encoding method used for encoding the three-dimensional data among the first encoding method and the second encoding method included in the control information (e.g., parameter set) of the encoding stream generated by encoding the three-dimensional data (e.g., a first encoding method and a second encoding method, e.g., identification information of a codec), and decodes the encoding stream using the determined encoding method.

[0161] According to this, when decoding a coded stream, the three-dimensional data decoder can determine the encoding method used for encoding by using information stored in the control information. Therefore, the three-dimensional data decoder can correctly decode the coded stream even when multiple encoding methods are used.

[0162] For example, the three-dimensional data includes location information, and the encoding stream includes encoded data of the location information. In the determination, the three-dimensional data decoding device determines the encoding method used for encoding the location information based on information indicating the encoding method used for encoding the location information among the first encoding method and the second encoding method, which is included in the control information of the location information included in the encoding stream. In the decoding, the three-dimensional data decoding device decodes the encoded data of the location information using the determined encoding method used for encoding the location information.

[0163] For example, the three-dimensional data includes location information and attribute information, and the encoding stream includes encoded data of the location information and encoded data of the attribute information. In the determination, the three-dimensional data decoding device determines the encoding method used for encoding the location information based on information indicating the encoding method used for encoding the location information among the first encoding method and the second encoding method included in the control information of the location information included in the encoding stream, and determines the encoding method used for encoding the attribute information based on information indicating the encoding method used for encoding the attribute information among the first encoding method and the second encoding method included in the control information of the attribute information included in the encoding stream. In the decoding, the three-dimensional data decoding device decodes the encoded data of the location information using the determined encoding method used for encoding the location information, and decodes the encoded data of the attribute information using the determined encoding method used for encoding the attribute information.

[0164] According to this, since different encoding methods can be used for location information and attribute information, encoding efficiency can be improved.

[0165] For example, the above-mentioned encoding stream is stored in one or more units (e.g., NAL units), and the three-dimensional data decoding device also acquires the above-mentioned encoding stream from the one or more units.

[0166] For example, the unit has a format common to the first encoding method and the second encoding method, and includes information that has a definition independent of the first encoding method and the second encoding method (e.g., pcc_nal_unit_type) as information indicating the type of data included in the unit.

[0167] For example, the unit has a format independent of the first encoding method and the second encoding method, and includes information indicating the type of data included in the unit, which has a definition independent of the first encoding method and the second encoding method (e.g., codec1_nal_unit_type or codec2_nal_unit_type).

[0168] For example, the unit has a format common to the first encoding method and the second encoding method, and includes information having a definition common to the first encoding method and the second encoding method (e.g., pcc_nal_unit_type) as information indicating the type of data included in the unit.

[0169] For example, a three-dimensional data decoding device is equipped with a processor and memory, and the processor uses the memory to perform the above processing.

[0170] (Embodiment 2)

[0171] In this embodiment, a method for saving NAL units to an ISOBMFF file is described.

[0172] ISOBMFF (ISO based media file format) is a file format specification defined in ISO / IEC 14496-12. ISOBMFF defines a format capable of storing various media, such as video, audio, and text, through multiplexing, and is a standard that is independent of the media.

[0173] This explains the basic structure (file) of ISOBMFF. The basic unit in ISOBMFF is the box. A box consists of type, length, and data, and a file is a collection of boxes of various types combined.

[0174] Figure 18 is a diagram showing the basic structure (file) of ISOBMFF. The ISOBMFF file mainly includes boxes such as ftyp, which represents the file brand in 4CC (4 character code), moov, which stores metadata such as control information, and mdat, which stores data.

[0175] Storage methods for ISOBMFF files for each medium are separately specified; for example, storage methods for AVC video and HEVC video are specified in ISO / IEC 14496-15. Here, one might consider extending the functions of ISOBMFF to accumulate or transmit PCC encoded data, but there are no regulations yet regarding storing PCC encoded data in ISOBMFF files. Therefore, in this embodiment, a method for storing PCC encoded data in an ISOBMFF file is described.

[0176] FIG. 19 is a diagram showing the protocol stack when a PCC codec common NAL unit is stored in an ISOBMFF file. Here, a PCC codec common NAL unit is stored in an ISOBMFF file. Although the NAL unit is PCC codec common, since multiple PCC codecs are stored in the NAL unit, it is desirable to define a storage method (Carriage of Codec1, Carriage of Codec2) according to each codec.

[0177] Next, a method for storing a common PCC NAL unit that supports multiple PCC codecs in an ISOBMFF file will be described. FIG. 20 is a diagram showing an example of storing a common PCC NAL unit in an ISOBMFF file of the storage method of Codec 1 (Carriage of Codec 1). FIG. 21 is a diagram showing an example of storing a common PCC NAL unit in an ISOBMFF file of the storage method of Codec 2 (Carriage of Codec 2).

[0178] Here, ftyp is important information for identifying the file format, and for ftyp, a different identifier is defined for each codec. When PCC encoded data encoded with the first encoding method (encoding scheme) is stored in a file, ftyp is set to pcc1. When PCC encoded data encoded with the second encoding method is stored in a file, ftyp is set to pcc2.

[0179] Here, pcc1 indicates that PCC codec 1 (first encoding method) is used. pcc2 indicates that PCC codec 2 (second encoding method) is used. That is, pcc1 and pcc2 indicate that the data is PCC (coded data of three-dimensional data (point cloud data)), and also represent PCC codecs (first encoding method and second encoding method).

[0180] The following describes how to save NAL units to an ISOBMFF file. The multiplexer interprets the NAL unit header, and if pcc_codec_type=Codec1, writes pcc1 to ftyp in ISOBMFF.

[0181] Also, the multiplexer interprets the NAL unit header, and if pcc_codec_type=Codec2, writes pcc2 to ftyp of ISOBMFF.

[0182] Additionally, if pcc_nal_unit_type is metadata, the multiplexer stores the NAL unit in a predetermined way, for example, in moov or mdat. If pcc_nal_unit_type is data, the multiplexer stores the NAL unit in a predetermined way, for example, in moov or mdat.

[0183] For example, the multiplexer may store the NAL unit size in the NAL unit, just like HEVC.

[0184] By the present storage method, by interpreting the ftyp included in the file in the demultiplexer (system layer), it becomes possible to determine whether PCC encoded data has been encoded by the first encoding method or the second encoding method. Furthermore, as described above, by determining whether PCC encoded data has been encoded by the first encoding method or the second encoding method, encoded data encoded by one of the encoding methods can be extracted from data containing a mixture of encoded data encoded by both encoding methods. As a result, the amount of data transmitted can be reduced when transmitting encoded data. In addition, by the present storage method, a common data format can be used in the first encoding method and the second encoding method without setting different data (file) formats.

[0185] In addition, if codec identification information appears in system layer metadata such as ftyp in ISOBMFF, the multiplexer may save the NAL unit with pcc_nal_unit_type removed to the ISOBMFF file.

[0186] Next, the configuration and operation of the multiplexer provided by the three-dimensional data encoding system (three-dimensional data encoding device) according to the present embodiment, and the demultiplexer provided by the three-dimensional data decoding system (three-dimensional data decoding device) according to the present embodiment will be described.

[0187] FIG. 22 is a diagram showing the configuration of the first multiplexer (4710). The first multiplexer (4710) includes a file conversion unit (4711) that generates multiplexed data (file) by storing the encoded data and control information (NAL unit) generated by the first encoding unit (4630) in an ISOBMFF file. This first multiplexer (4710) is included, for example, in the multiplexer (4614) shown in FIG. 1.

[0188] FIG. 23 is a diagram showing the configuration of a first demultiplexer (4720). The first demultiplexer (4720) has a file inverse conversion unit (4721) that acquires encoded data and control information (NAL unit) from multiplexed data (file) and outputs the acquired encoded data and control information to a first decoding unit (4640). This first demultiplexer (4720) is included in, for example, the demultiplexer (4623) shown in FIG. 1.

[0189] FIG. 24 is a diagram showing the configuration of a second multiplexer (4730). The second multiplexer (4730) includes a file conversion unit (4731) that generates multiplexed data (file) by storing the encoded data and control information (NAL unit) generated by the second encoding unit (4650) in an ISOBMFF file. This second multiplexer (4730) is included, for example, in the multiplexer (4614) shown in FIG. 1.

[0190] FIG. 25 is a diagram showing the configuration of a second demultiplexer (4740). The second demultiplexer (4740) includes a file inverse conversion unit (4741) that acquires encoded data and control information (NAL unit) from multiplexed data (file) and outputs the acquired encoded data and control information to a second decoding unit (4660). This second demultiplexer (4740) is included in, for example, the demultiplexer (4623) shown in FIG. 1.

[0191] FIG. 26 is a flowchart of multiplexing processing by the first multiplexer (4710). First, the first multiplexer (4710) determines whether the codec being used is the first encoding method or the second encoding method by interpreting the pcc_codec_type included in the NAL unit header (S4701).

[0192] If pcc_codec_type indicates a second encoding method (second encoding method in S4702), the first multiplexer (4710) does not process the NAL unit (S4703).

[0193] Meanwhile, if pcc_codec_type indicates a second encoding method (the first encoding method in S4702), the first multiplexer (4710) writes pcc1 in ftyp (S4704). That is, the first multiplexer (4710) writes information in ftyp indicating that data encoded by the first encoding method is stored in a file.

[0194] Next, the first multiplexer (4710) interprets the pcc_nal_unit_type included in the NAL unit header and stores the data in a box (moov or mdat, etc.) in a predetermined way according to the data type appearing in the pcc_nal_unit_type (S4705). Then, the first multiplexer (4710) creates an ISOBMFF file including the ftyp and the box (S4706).

[0195] FIG. 27 is a flowchart of multiplexing processing by the second multiplexer (4730). First, the second multiplexer (4730) determines whether the codec being used is the first encoding method or the second encoding method by interpreting the pcc_codec_type included in the NAL unit header (S4711).

[0196] When pcc_unit_type indicates a second encoding method (second encoding method in S4712), the second multiplexer (4730) writes pcc2 in ftyp (S4713). That is, the second multiplexer (4730) writes information in ftyp indicating that data encoded by the second encoding method is stored in a file.

[0197] Next, the second multiplexer (4730) interprets the pcc_nal_unit_type included in the NAL unit header and stores the data in a box (moov or mdat, etc.) in a predetermined way according to the data type appearing in the pcc_nal_unit_type (S4714). Then, the second multiplexer (4730) creates an ISOBMFF file including the ftyp and the box (S4715).

[0198] Meanwhile, if pcc_unit_type represents the first encoding method (the first encoding method in S4712), the second multiplexer (4730) does not process the NAL unit (S4716).

[0199] Additionally, the above processing illustrates an example of encoding PCC data using either the first encoding method or the second encoding method. The first multiplexer (4710) and the second multiplexer (4730) save the desired NAL unit to a file by identifying the codec type of the NAL unit. Furthermore, if identification information of the PCC codec is included in addition to the NAL unit header, the first multiplexer (4710) and the second multiplexer (4730) may identify the codec type (the first encoding method or the second encoding method) by using the identification information of the PCC codec included in addition to the NAL unit header in steps S4701 and S4711.

[0200] Additionally, the first multiplexer (4710) and the second multiplexer (4730) may save the data to the file after deleting pcc_nal_unit_type from the NAL unit header when saving the data to the file in steps S4706 and S4714.

[0201] FIG. 28 is a flowchart illustrating processing by the first demultiplexer (4720) and the first decoding unit (4640). First, the first demultiplexer (4720) interprets ftyp included in the ISOBMFF file (S4721). If the codec appearing in ftyp is the second encoding method (pcc2) (second encoding method in S4722), the first demultiplexer (4720) determines that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4723). Also, the first demultiplexer (4720) transmits the result of the determination to the first decoding unit (4640). The first decoding unit (4640) does not process the NAL unit (S4724).

[0202] Meanwhile, if the codec appearing in ftyp is the first encoding method (pcc1) (the first encoding method in S4722), the first demultiplexer (4720) determines that the data included in the payload of the NAL unit is data encoded by the first encoding method (S4725). Also, the first demultiplexer (4720) transmits the result of the determination to the first decoding unit (4640).

[0203] The first decoding unit (4640) identifies data by determining that the pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first encoding method (S4726). Then, the first decoding unit (4640) decodes the PCC data using the decoding process of the first encoding method (S4727).

[0204] FIG. 29 is a flowchart illustrating processing by the second demultiplexer (4740) and the second decoding unit (4660). First, the second demultiplexer (4740) interprets ftyp included in the ISOBMFF file (S4731). If the codec appearing in ftyp is the second encoding method (pcc2) (the second encoding method in S4732), the second demultiplexer (4740) determines that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4733). Also, the second demultiplexer (4740) transmits the result of the determination to the second decoding unit (4660).

[0205] The second decoding unit (4660) identifies data by determining that the pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the second encoding method (S4734). Then, the second decoding unit (4660) decodes the PCC data using the decoding process of the second encoding method (S4735).

[0206] Meanwhile, if the codec appearing in ftyp is the first encoding method (pcc1) (the first encoding method in S4732), the second demultiplexer (4740) determines that the data included in the payload of the NAL unit is data encoded by the first encoding method (S4736). Also, the second demultiplexer (4740) transmits the result of the determination to the second decoding unit (4660). The second decoding unit (4660) does not process the NAL unit (S4737).

[0207] In this way, for example, in the first demultiplexer (4720) or the second demultiplexer (4740), the codec type can be identified at an early stage by identifying the codec type of the NAL unit. Additionally, the desired NAL unit can be input into the first decoding unit (4640) or the second decoding unit (4660), and unnecessary NAL units can be removed. In this case, the processing of interpreting the codec identification information in the first decoding unit (4640) or the second decoding unit (4660) may become unnecessary. Furthermore, the processing of interpreting the codec identification information by referring to the NAL unit type again in the first decoding unit (4640) or the second decoding unit (4660) may be performed.

[0208] Additionally, if the pcc_nal_unit_type is deleted from the NAL unit header in the first multiplexer (4710) or the second multiplexer (4730), the first demultiplexer (4720) or the second demultiplexer (4740) may assign the pcc_nal_unit_type to the NAL unit and then output it to the first decoding unit (4640) or the second decoding unit (4660).

[0209] (Embodiment 3)

[0210] In this embodiment, a multiplexer and a demultiplexer corresponding to the encoding unit (4670) and the decoding unit (4680) corresponding to the plurality of codecs described in Embodiment 1 are described. FIG. 30 is a diagram showing the configuration of the encoding unit (4670) and the third multiplexer (4750) according to this embodiment.

[0211] The encoding unit (4670) encodes point cloud data using either or both of the first encoding method and the second encoding method. The encoding unit (4670) may switch the encoding method (first encoding method and second encoding method) on a point cloud data unit or a frame unit. Additionally, the encoding unit (4670) may switch the encoding method on a coding-capable unit.

[0212] The encoding unit (4670) generates encoding data (encoding stream) containing identification information of the PCC codec.

[0213] The third multiplexer (4750) is equipped with a file conversion unit (4751). The file conversion unit (4751) converts the NAL unit output from the encoding unit (4670) into a PCC data file. The file conversion unit (4751) interprets the codec identification information included in the NAL unit header and determines whether the PCC encoded data is data encoded by the first encoding method, data encoded by the second encoding method, or data encoded by both methods. The file conversion unit (4751) writes a brand name that can identify the codec in ftyp. For example, if it indicates that it is encoded by both methods, pcc3 is written in ftyp.

[0214] Additionally, if the encoding unit (4670) includes identification information of a PCC codec in addition to the NAL unit, the file conversion unit (4751) may determine the PCC codec (encoding method) using said identification information.

[0215] FIG. 31 is a diagram showing the configuration of the third demultiplexer (4760) and the decoding unit (4680) according to the present embodiment.

[0216] The third demultiplexer (4760) is provided with a file inverse conversion unit (4761). The file inverse conversion unit (4761) interprets ftyp included in the file and determines whether the PCC encoded data is data encoded by the first encoding method, data encoded by the second encoding method, or data encoded by both methods.

[0217] When PCC encoded data is encoded using one of the encoding methods, data is input to the corresponding decoding unit among the first decoding unit (4640) and the second decoding unit (4660), and data is not input to the other decoding unit. When PCC encoded data is encoded using both encoding methods, data is input to the decoding unit (4680) corresponding to both methods.

[0218] The decoding unit (4680) decodes the PCC encoded data using either one or both of the first encoding method and the second encoding method.

[0219] FIG. 32 is a flowchart showing processing by the third multiplexer (4750) according to the present embodiment.

[0220] First, the third multiplexer (4750) determines whether the codec being used is the first encoding method, the second encoding method, or both the first encoding method and the second encoding method by interpreting the pcc_codec_type included in the NAL unit header (S4741).

[0221] When the second encoding method is being used (Yes in S4742, and also the second encoding method in S4743), the third multiplexer (4750) writes pcc2 in ftyp (S4744). That is, the third multiplexer (4750) writes information in ftyp indicating that data encoded by the second encoding method is stored in a file.

[0222] Next, the third multiplexer (4750) interprets the pcc_nal_unit_type included in the NAL unit header and stores the data in a box (moov or mdat, etc.) in a predetermined way according to the data type appearing in the pcc_unit_type (S4745). Then, the third multiplexer (4750) creates an ISOBMFF file including the ftyp and the box (S4746).

[0223] Meanwhile, if the first encoding method is being used (Yes in S4742, and also the first encoding method in S4743), the third multiplexer (4750) writes pcc1 in ftyp (S4747). That is, the third multiplexer (4750) writes information in ftyp indicating that data encoded by the first encoding method is stored in a file.

[0224] Next, the third multiplexer (4750) interprets the pcc_nal_unit_type included in the NAL unit header and stores the data in a box (moov or mdat, etc.) in a predetermined way according to the data type appearing in the pcc_unit_type (S4748). Then, the third multiplexer (4750) creates an ISOBMFF file including the ftyp and the box (S4746).

[0225] Meanwhile, when both the first encoding method and the second encoding method are used (No in S4742), the third multiplexer (4750) writes pcc3 in ftyp (S4749). That is, the third multiplexer (4750) writes information in ftyp indicating that data encoded by both encoding methods is stored in a file.

[0226] Next, the third multiplexer (4750) interprets the pcc_nal_unit_type included in the NAL unit header and stores the data in a box (moov or mdat, etc.) in a predetermined way according to the data type appearing in the pcc_unit_type (S4750). Then, the third multiplexer (4750) creates an ISOBMFF file including the ftyp and the box (S4746).

[0227] FIG. 33 is a flowchart illustrating processing by the third demultiplexer (4760) and the decoding unit (4680). First, the third demultiplexer (4760) interprets the ftyp included in the ISOBMFF file (S4761). If the codec appearing in the ftyp is the second encoding method (pcc2) (Yes in S4762, and the second encoding method in S4763), the third demultiplexer (4760) determines that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4764). Also, the third demultiplexer (4760) transmits the result of the determination to the decoding unit (4680).

[0228] The decoding unit (4680) identifies the data by determining that the pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the second encoding method (S4765). Then, the decoding unit (4680) decodes the PCC data using the decoding process of the second encoding method (S4766).

[0229] Meanwhile, if the codec appearing in ftyp is the first encoding method (pcc1) (Yes in S4762, and also the first encoding method in S4763), the third demultiplexer (4760) determines that the data included in the payload of the NAL unit is data encoded by the first encoding method (S4767). Also, the third demultiplexer (4760) transmits the result of the determination to the decoder (4680).

[0230] The decoding unit (4680) identifies the data by determining that the pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first encoding method (S4768). Then, the decoding unit (4680) decodes the PCC data using the decoding process of the first encoding method (S4769).

[0231] Meanwhile, if it appears that both encoding methods are being used in ftyp (pcc3) (No in S4762), the third demultiplexer (4760) determines that the data included in the payload of the NAL unit is data encoded using both the first encoding method and the second encoding method (S4770). Also, the third demultiplexer (4760) transmits the result of the determination to the decoding unit (4680).

[0232] The decoding unit (4680) identifies data by determining that the pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the codec listed in the pcc_codec_type (S4771). Then, the decoding unit (4680) decodes the PCC data using the decoding processing of both encoding methods (S4772). That is, the decoding unit (4680) decodes the data encoded by the first encoding method using the decoding processing of the first encoding method, and decodes the data encoded by the second encoding method using the decoding processing of the second encoding method.

[0233] Variations of the present embodiment are described below. As for the types of brands appearing in ftyp, the following types may appear as identification information. In addition, combinations of multiple types shown below may appear as identification information.

[0234] The identification information may indicate whether the object of the original data prior to PCC encoding is a point cloud with a limited area or a large point cloud without a limited area, such as map information.

[0235] The identification information may indicate whether the original data prior to PCC encoding is a static object or a dynamic object.

[0236] As described above, the identification information may indicate whether the PCC encoded data is data encoded by the first encoding method or data encoded by the second encoding method.

[0237] The identification information may represent the algorithm used in PCC encoding. Here, the algorithm is, for example, an encoding method usable in the first encoding method or the second encoding method.

[0238] The identification information may indicate the difference in the storage method for the ISOBMFF file of PCC encoded data. For example, the identification information may indicate whether the storage method used is a storage method for archiving or a storage method for real-time transmission, such as dynamic streaming.

[0239] In addition, in Embodiments 2 and 3, examples in which ISOBMFF is used as a file format were described, but other methods may be used. For example, when storing PCC encoded data in MPEG-2 TS Systems, MPEG-DASH, MMT, or RMP, the same method as in the present embodiment may be used.

[0240] In addition, the above description shows an example of storing metadata, such as identification information, in ftyp, but such metadata may be stored in a place other than ftyp. For example, such metadata may be stored in moov.

[0241] As described above, the three-dimensional data storage device (or three-dimensional data multiplexing device, or three-dimensional data encoding device) performs the processing shown in FIG. 34.

[0242] First, a three-dimensional data storage device (e.g., including a first multiplexer (4710), a second multiplexer (4730), or a third multiplexer (4750)) acquires one or more units (e.g., NAL units) in which a coded stream of point cloud data is encoded is stored (S4781). Next, the three-dimensional data storage device stores one or more units in a file (e.g., a file of ISOBMFF) (S4782). In addition, in the storage (S4782), the three-dimensional data storage device stores information (e.g., pcc1, pcc2, or pcc3) indicating that the data stored in the file is data encoded with point cloud data in the control information (e.g., ftyp) of the file.

[0243] According to this, a device processing a file generated by the said three-dimensional data storage device can determine early whether the data stored in the said file is encoded data of point cloud data by referring to the file's control information. Accordingly, it is possible to reduce the throughput of the said device or accelerate the processing.

[0244] For example, the above information also indicates the encoding method used for encoding the point cloud data among the first encoding method and the second encoding method. Additionally, the fact that the data stored in the file is data encoded from the point cloud data, and the encoding method used for encoding the point cloud data among the first encoding method and the second encoding method, may be represented as a single piece of information or as different pieces of information.

[0245] According to this, a device processing a file generated by the said three-dimensional data storage device can determine early on the codec used for the data stored in the said file by referring to the file's control information. Accordingly, it is possible to reduce the throughput of the said device or accelerate the processing.

[0246] For example, the first encoding method is a method (GPCC) that encodes position information representing the location of point cloud data as an N-ary tree (N is an integer greater than or equal to 2) and encodes attribute information using the position information, and the second encoding method is a method (VPCC) that generates a two-dimensional image from point cloud data and encodes the two-dimensional image using an image encoding method.

[0247] For example, the above file conforms to ISOBMFF (ISO based media file format).

[0248] For example, a three-dimensional data storage device is equipped with a processor and memory, and the processor uses the memory to perform the above processing.

[0249] In addition, as described above, the three-dimensional data acquisition device (or three-dimensional data demultiplexing device or three-dimensional data decoding device) performs the processing shown in FIG. 35.

[0250] A three-dimensional data acquisition device (including, for example, a first demultiplexer (4720), a second demultiplexer (4740), or a third demultiplexer (4760)) acquires a file (for example, an ISOBMFF file) in which one or more units (for example, NAL units) are stored, in which a encoded stream in which point cloud data is encoded is stored (S4791). Next, the three-dimensional data acquisition device acquires one or more units from the file (S4792). In addition, the control information of the file (for example, ftyp) includes information (for example, pcc1, pcc2, or pcc3) indicating that the data stored in the file is data in which point cloud data is encoded.

[0251] For example, the three-dimensional data acquisition device determines whether the data stored in the file is data encoded with point cloud data by referring to the above information. Also, if the three-dimensional data acquisition device determines that the data stored in the file is data encoded with point cloud data, it generates point cloud data by decoding the data encoded with point cloud data included in one or more units. Or, if the three-dimensional data acquisition device determines that the data stored in the file is data encoded with point cloud data, it outputs (notifies) information indicating that the data included in one or more units is data encoded with point cloud data to a subsequent processing unit (for example, a first decoding unit (4640), a second decoding unit (4660), or a decoding unit (4680)).

[0252] According to this, the three-dimensional data acquisition device can determine early whether the data stored in the file is encoded data of point cloud data by referring to the control information of the file. Accordingly, it is possible to reduce the throughput of the three-dimensional data acquisition device or the downstream device or to accelerate the processing.

[0253] For example, the above information also indicates the encoding method used for encoding among the first encoding method and the second encoding method. Additionally, the fact that the data stored in the file is data encoded from point cloud data, and the encoding method used for encoding the point cloud data among the first encoding method and the second encoding method, may be represented as a single piece of information or as different pieces of information.

[0254] According to this, the three-dimensional data acquisition device can determine early on the codec used for the data stored in the file by referring to the file control information. Accordingly, it is possible to reduce the throughput of the three-dimensional data acquisition device or the subsequent device, or to accelerate the processing.

[0255] For example, a three-dimensional data acquisition device acquires data encoded by one of the encoding methods from encoded point cloud data including data encoded by a first encoding method and data encoded by a second encoding method, based on the above information.

[0256] For example, the first encoding method is a method (GPCC) that encodes position information representing the location of point cloud data as an N-ary tree (N is an integer greater than or equal to 2) and encodes attribute information using the position information, and the second encoding method is a method (VPCC) that generates a two-dimensional image from point cloud data and encodes the two-dimensional image using an image encoding method.

[0257] For example, the above file conforms to ISOBMFF (ISO based media file format).

[0258] For example, a three-dimensional data acquisition device is equipped with a processor and memory, and the processor uses the memory to perform the above processing.

[0259] (Embodiment 4)

[0260] In this embodiment, the types of encoded data (position information (Geometry), attribute information (Attribute), additional information (Metadata)) generated by the first encoding unit (4630) or the second encoding unit (4650) described above, the method for generating additional information (metadata), and multiple processing in the multiplexing unit are explained. Additionally, additional information (metadata) may be referred to as a parameter set or control information.

[0261] In this embodiment, the dynamic object (time-varying three-dimensional point cloud data) described in FIG. 4 is used as an example, but the same method may be used for static objects (three-dimensional point cloud data at any given time).

[0262] FIG. 36 is a diagram showing the configuration of an encoding unit (4801) and a multiplexing unit (4802) included in a three-dimensional data encoding device according to the present embodiment. The encoding unit (4801) corresponds, for example, to the first encoding unit (4630) or the second encoding unit (4650) described above. The multiplexing unit (4802) corresponds to the multiplexing unit (4634 or 4656) described above.

[0263] The encoding unit (4801) encodes point cloud data of multiple PCC (Point Cloud Compression) frames and generates multiple compressed data of multiple location information, attribute information and additional information.

[0264] The multiplexing unit (4802) converts data of multiple data types (location information, attribute information and additional information) into a data configuration that takes into account data access in a decoding device by converting the data into NAL units.

[0265] FIG. 37 is a diagram showing an example of the configuration of encoded data generated by the encoding unit (4801). The arrows in the diagram represent dependency relationships according to the decoding of the encoded data, and the root of the arrow depends on the data at the end of the arrow. That is, the decoding device decodes the data at the end of the arrow and uses the decoded data to decode the data at the root of the arrow. In other words, dependency means that the data of the dependent source is referenced (used) in the processing (encoding or decoding, etc.) of the dependent source's data.

[0266] First, the process of generating encoded data of location information is explained. The encoding unit (4801) generates encoded location data (Compressed Geometry Data) for each frame by encoding the location information of each frame. Also, the encoded location data is denoted as G(i). i represents the frame number or the time of the frame, etc.

[0267] Additionally, the encoding unit (4801) generates a set of position parameters (GPS(i)) corresponding to each frame. The set of position parameters includes parameters that can be used to decode encoded position data. Also, the encoded position data for each frame depends on the corresponding set of position parameters.

[0268] Additionally, encoded position data consisting of multiple frames is defined as a position sequence (Geometry Sequence). The encoding unit (4801) generates a position sequence parameter set (Geometry Sequence PS: also referred to as Position SPS) that stores parameters commonly used for decoding processing for multiple frames within the position sequence. The position sequence depends on the Position SPS.

[0269] Next, the process of generating encoded data of attribute information is described. The encoding unit (4801) generates compressed attribute data for each frame by encoding the attribute information of each frame. Also, the compressed attribute data is denoted as A(i). Also, FIG. 37 shows an example in which attribute X and attribute Y exist, the compressed attribute data of attribute X is denoted as AX(i), and the compressed attribute data of attribute Y is denoted as AY(i).

[0270] Additionally, the encoding unit (4801) generates an attribute parameter set (APS(i)) corresponding to each frame. Also, the attribute parameter set of attribute X is denoted as AXPS(i), and the attribute parameter set of attribute Y is denoted as AYPS(i). The attribute parameter set includes parameters that can be used for decoding encoding attribute information. The encoding attribute data depends on the corresponding attribute parameter set.

[0271] Additionally, encoding attribute data consisting of multiple frames is defined as an attribute sequence. The encoding unit (4801) generates an attribute sequence parameter set (Attribute Sequence PS: also referred to as Attribute SPS) that stores parameters commonly used for decoding processing for multiple frames within the attribute sequence. The attribute sequence depends on the Attribute SPS.

[0272] Also, in the first encoding method, the encoding attribute data depends on the encoding position data.

[0273] Also, FIG. 37 illustrates an example where there are two types of attribute information (attribute X and attribute Y). When there are two types of attribute information, for example, data and metadata are generated by two encoding units. Also, for example, an attribute sequence is defined for each type of attribute information, and an attribute SPS is generated for each type of attribute information.

[0274] Additionally, FIG. 37 illustrates an example where there is one type of position information and two types of attribute information, but it is not limited to this; the attribute information may be one type or three or more types. In this case, encoded data can be generated in the same way. Also, in the case of point cloud data that does not have attribute information, attribute information may not be present. In that case, the encoding unit (4801) does not need to generate a parameter set related to attribute information.

[0275] Next, the process of generating additional information (metadata) will be described. The encoding unit (4801) generates a PCC stream PS (PCC Stream PS: also referred to as stream PS), which is a set of parameters for the entire PCC stream. The encoding unit (4801) stores parameters in stream PS that can be commonly used for decoding processing for one or more position sequences and one or more attribute sequences. For example, stream PS includes identification information indicating the codec of the point cloud data and information indicating the algorithm used for encoding. The position sequence and attribute sequence depend on stream PS.

[0276] Next, the access unit and GOF will be described. In this embodiment, the concepts of an access unit (Access Unit: AU) and a GOF (Group of Frame) are newly introduced.

[0277] An access unit is a basic unit for accessing data during decoding and consists of one or more data and one or more metadata. For example, an access unit consists of location information at the same time and one or more attribute information. A GOF is a random access unit and consists of one or more access units.

[0278] The encoding unit (4801) generates an access unit header (AU Header) as identification information indicating the beginning of the access unit. The encoding unit (4801) stores parameters related to the access unit in the access unit header. For example, the access unit header includes the configuration or information of the encoded data included in the access unit. In addition, the access unit header includes parameters commonly used for the data included in the access unit, such as parameters related to the decoding of the encoded data.

[0279] Additionally, the encoding unit (4801) may generate an access unit delimiter that does not contain parameters related to the access unit, instead of an access unit header. This access unit delimiter is used as identification information indicating the beginning of the access unit. The decoding device identifies the beginning of the access unit by detecting the access unit header or the access unit delimiter.

[0280] Next, the generation of identification information for the GOF head is described. The encoding unit (4801) generates a GOF header as identification information indicating the GOF head. The encoding unit (4801) stores parameters related to the GOF in the GOF header. For example, the GOF header includes the composition or information of the encoded data included in the GOF. In addition, the GOF header includes parameters commonly used in the data included in the GOF, such as parameters related to the decoding of the encoded data.

[0281] Additionally, the encoding unit (4801) may generate a GOF delimiter that does not contain parameters related to the GOF instead of a GOF header. This GOF delimiter is used as identification information indicating the beginning of the GOF. The decoding device identifies the beginning of the GOF by detecting the GOF header or the GOF delimiter.

[0282] In PCC encoded data, for example, an access unit is defined as a PCC frame unit. A decoder accesses a PCC frame based on the identification information at the beginning of the access unit.

[0283] Also, for example, GOF is defined as a single random access unit. The decoder accesses the random access unit based on the identification information at the beginning of the GOF. For example, if PCC frames can be decoded independently without interdependencies, PCC frames may be defined as random access units.

[0284] In addition, two or more PCC frames may be allocated to one access unit, and multiple random access units may be allocated to one GOF.

[0285] Additionally, the encoding unit (4801) may define and generate parameter sets or metadata other than those mentioned above. For example, the encoding unit (4801) may generate SEI (Supplemental Enhancement Information) that stores parameters (optional parameters) that may not necessarily be used during decoding.

[0286] Next, the structure of the encoded data and the method of storing the encoded data in the NAL unit will be explained.

[0287] For example, a data format is defined for each type of encoded data. FIG. 38 is a diagram showing examples of encoded data and NAL units.

[0288] For example, as shown in FIG. 38, the encoded data includes a header and a payload. Additionally, the encoded data may include length information indicating the length (amount of data) of the encoded data, the header, or the payload. Also, the encoded data may not include a header.

[0289] A header contains identification information to identify data, for example. This identification information indicates, for example, the data type or frame number.

[0290] A header contains identification information indicating a reference relationship, for example. This identification information is stored in the header when there is a dependency relationship between data, for example, and is information for referencing a reference location from a reference source. For example, the header of a reference location contains identification information to identify the data in question. The header of a reference source contains identification information indicating the reference location.

[0291] In addition, if the reference site or source can be identified or derived from other information, identification information for specifying data or identification information indicating a reference relationship may be omitted.

[0292] The multiplexer (4802) stores the encoded data in the payload of the NAL unit. The NAL unit header includes pcc_nal_unit_type, which is identification information of the encoded data. FIG. 39 is a diagram showing an example of the semantics of pcc_nal_unit_type.

[0293] As shown in FIG. 39, when pcc_codec_type is codec 1 (Codec1: first encoding method), values ​​0 to 10 of pcc_nal_unit_type are assigned to the encoding position data (Geometry), encoding attribute X data (AttributeX), encoding attribute Y data (AttributeY), position PS (Geom. PS), attribute XPS (AttrX. PS), attribute YPS (AttrX. PS), position SPS (Geometry Sequence PS), attribute XSPS (AttributeX Sequence PS), attribute YSPS (AttributeY Sequence PS), AU header (AU Header), and GOF header (GOF Header) in codec 1. Also, values ​​11 and above are assigned as reserves for codec 1.

[0294] When pcc_codec_type is Codec 2 (Codec2: second encoding method), values ​​0 to 2 of pcc_nal_unit_type are assigned to the codec's data A (DataA), metadata A (MetaDataA), and metadata B (MetaDataB). Also, values ​​3 and above are assigned as reserves for Codec 2.

[0295] Next, the data transmission order will be explained. Below, the constraints on the transmission order of the NAL unit will be explained.

[0296] The multiplexer (4802) groups NAL units into GOF or AU units and transmits them. The multiplexer (4802) places a GOF header at the beginning of the GOF and places an AU header at the beginning of the AU.

[0297] Even if data is lost due to packet loss, etc., the multiplexer (4802) may place a sequence parameter set (SPS) for each AU so that the decoding device can decode from the next AU.

[0298] If there is a dependency relationship between the encoded data and the decoding, the decoding device decodes the data of the reference source after decoding the data of the reference location. In order to enable the decoding device to decode in the received order without rearranging the data, the multiplexer (4802) first transmits the data of the reference location.

[0299] FIG. 40 is a diagram showing an example of the transmission sequence of a NAL unit. FIG. 40 shows three examples of location information priority, parameter priority, and data integration.

[0300] The location-first transmission order is an example where location information and attribute information are bundled and transmitted separately. In this transmission order, the transmission of location information is completed faster than the transmission of attribute information.

[0301] For example, by using this transmission sequence, a decoding device that does not decode attribute information may be able to save time by ignoring the decoding of attribute information. Also, for example, a decoding device that wants to decode position information quickly may be able to decode position information faster by obtaining the encoded data of position information quickly.

[0302] In addition, in FIG. 40, the attribute XSPS and attribute YSPS are combined and listed as the attribute SPS, but the attribute XSPS and attribute YSPS may be placed individually.

[0303] In a parameter set priority transmission order, the parameter set is transmitted first, and the data is transmitted next.

[0304] According to the constraints on the transmission order of NAL units as described above, the multiplexer (4802) may transmit NAL units in any order. For example, sequence identification information may be defined, and the multiplexer (4802) may have the function of transmitting NAL units in the order of multiple patterns. For example, the sequence identification information of NAL units is stored in stream PS.

[0305] The three-dimensional data decoding device may perform decoding based on sequence identification information. A desired transmission sequence is indicated from the three-dimensional data decoding device to the three-dimensional data encoding device, and the three-dimensional data encoding device (multiplexer (4802)) may control the transmission sequence according to the indicated transmission sequence.

[0306] Additionally, the multiplexer (4802) may generate encoded data that merges multiple functions, provided that the range is within the constraints of the transmission order, such as the transmission order of data integration. For example, as shown in FIG. 40, the GOF header and the AU header may be integrated, or the AXPS and AYPS may be integrated. In this case, an identifier indicating that the data has multiple functions is defined in pcc_nal_unit_type.

[0307] Variations of the present embodiment are described below. The PS has levels, such as frame-level PS, sequence-level PS, and PCC sequence-level PS. If the PCC sequence level is set as the upper level and the frame level as the lower level, the following method may be used for storing parameters.

[0308] The default PS value is displayed in the higher-level PS. Also, if the value of the lower-level PS is different from the value of the higher-level PS, the PS value is displayed in the lower-level PS. Alternatively, the PS value is not listed in the higher-level PS, but is listed in the lower-level PS. Alternatively, information on whether to display the PS value in the lower-level PS, the higher-level PS, or both is displayed in either the lower-level PS or the higher-level PS, or both. Alternatively, the lower-level PS may be merged into the higher-level PS. Alternatively, if the lower-level PS and the higher-level PS overlap, the multiplexer (4802) may omit the transmission of either one.

[0309] Additionally, the encoding unit (4801) or the multiplexing unit (4802) may divide the data into slices or tiles, and transmit the divided data. The divided data includes information for identifying the divided data, and parameters used for decoding the divided data are included in the parameter set. In this case, pcc_nal_unit_type is defined with an identifier indicating that it is data that stores data or parameters related to the tile or slice.

[0310] Below, processing according to sequence identification information is described. FIG. 41 is a flowchart of processing by a three-dimensional data encoding device (encoding unit (4801) and multiplexing unit (4802)) according to the transmission order of the NAL unit.

[0311] First, the three-dimensional data encoding device determines the transmission order of the NAL unit (position information priority or parameter set priority) (S4801). For example, the three-dimensional data encoding device determines the transmission order based on a designation from a user or an external device (e.g., a three-dimensional data decoding device).

[0312] If the determined transmission order is position information priority (position information priority in S480), the three-dimensional data encoding device sets the order identification information included in the stream PS to position information priority (S4803). That is, in this case, the order identification information indicates that NAL units are transmitted in the order of position information priority. Then, the three-dimensional data encoding device transmits NAL units in the order of position information priority (S4804).

[0313] Meanwhile, if the determined transmission order is parameter set priority (parameter set priority in S4802), the three-dimensional data encoding device sets the order identification information included in the stream PS to parameter set priority (S4805). That is, in this case, the order identification information indicates that NAL units are transmitted in the order of parameter set priority. Then, the three-dimensional data encoding device transmits NAL units in the order of parameter set priority (S4806).

[0314] FIG. 42 is a flowchart of processing by a three-dimensional data decoder according to the transmission order of the NAL unit. First, the three-dimensional data decoder interprets the sequence identification information included in the stream PS (S4811).

[0315] If the transmission order indicated in the sequence identification information is position information priority (position information priority in S4812), the three-dimensional data decoding device decodes the NAL unit by assuming that the transmission order of the NAL unit is position information priority (S4813).

[0316] Meanwhile, if the transmission order indicated in the sequence identification information is parameter set priority (parameter set priority in S4812), the three-dimensional data decoder decodes the NAL unit by assuming that the transmission order of the NAL unit is parameter set priority (S4814).

[0317] For example, if the three-dimensional data decoding device does not decode attribute information, in step S4813, it may not acquire all NAL units, but instead acquire NAL units related to position information and decode the position information from the acquired NAL units.

[0318] Next, processing according to the generation of AU and GOF will be described. FIG. 43 is a flowchart of processing by a three-dimensional data encoding device (multiplexer (4802)) according to the generation of AU and GOF in the multiplexing of NAL units.

[0319] First, the three-dimensional data encoding device determines the type of encoded data (S4821). Specifically, the three-dimensional data encoding device determines whether the encoded data to be processed is data leading from AU, data leading from GOF, or other data.

[0320] If the encoded data is the GOF leading data (GOF leading in S4822), the three-dimensional data encoding device creates a NAL unit by placing the GOF header and the AU header at the beginning of the encoded data belonging to the GOF (S4823).

[0321] If the encoded data is the data at the beginning of the AU (the AU beginning in S4822), the three-dimensional data encoding device creates a NAL unit by placing an AU header at the beginning of the encoded data belonging to the AU (S4824).

[0322] If the encoded data is not a GOF header or an AU header (in S4822, not a GOF header or an AU header), the three-dimensional data encoding device creates a NAL unit by placing the encoded data after the AU header of the AU to which the encoded data belongs (S4825).

[0323] Next, the processing according to access to the AU and GOF will be described. FIG. 44 is a flowchart of the processing of a three-dimensional data decoder according to access to the AU and GOF in the demultiplexing of the NAL unit.

[0324] First, the three-dimensional data decoder determines the type of encoded data included in the NAL unit by interpreting the nal_unit_type included in the NAL unit (S4831). Specifically, the three-dimensional data decoder determines whether the encoded data included in the NAL unit is data leading to AU, data leading to GOF, or other data.

[0325] When the encoded data included in the NAL unit is the data at the beginning of the GOF (the beginning of the GOF in S4832), the three-dimensional data decoding device determines that the NAL unit is the starting position for random access, accesses the NAL unit, and starts decoding processing (S4833).

[0326] Meanwhile, if the encoded data included in the NAL unit is the data leading to the AU (the AU leading in S4832), the three-dimensional data decoding device determines that the NAL unit is the AU leading, accesses the data included in the NAL unit, and decodes the said AU (S4834).

[0327] Meanwhile, if the encoded data included in the NAL unit is neither the GOF lead nor the AU lead (other than the GOF lead and AU lead in S4832), the three-dimensional data decoder does not process the NAL unit.

[0328] As described above, the three-dimensional data encoding device performs the processing shown in FIG. 45. The three-dimensional data encoding device encodes time-series three-dimensional data (e.g., point cloud data of a dynamic object). The three-dimensional data includes position information and attribute information for each time step.

[0329] First, the three-dimensional data encoding device encodes position information (S4841). Next, the three-dimensional data encoding device encodes attribute information of a processing target by referring to position information at the same time as the attribute information of the processing target (S4842). Here, as shown in FIG. 37, the position information and attribute information at the same time constitute an access unit (AU). That is, the three-dimensional data encoding device encodes attribute information of a processing target by referring to position information included in the same access unit as the attribute information of the processing target.

[0330] According to this, the three-dimensional data encoding device can facilitate the control of references during encoding by using an access unit. Accordingly, the three-dimensional data encoding device can reduce the throughput of the encoding process.

[0331] For example, a three-dimensional data encoding device generates a bit stream including encoded position information (encoded position data), encoded attribute information (encoded attribute data), and information indicating the position information of a reference location of the attribute information of the processing target.

[0332] For example, the bit stream includes a set of position parameters (position PS) containing control information for position information at each time point, and a set of attribute parameters (attribute PS) containing control information for attribute information at each time point.

[0333] For example, the bit stream includes a position sequence parameter set (position SPS) containing common control information for position information of multiple times, and an attribute sequence parameter set (attribute SPS) containing common control information for attribute information of multiple times.

[0334] For example, a bit stream includes a set of stream parameters (stream PS) containing control information common to position information of multiple times and attribute information of multiple times.

[0335] For example, the bit stream includes an access unit header (AU header) containing common control information within the access unit.

[0336] For example, a three-dimensional data encoding device encodes a GOF (Group of Frames) composed of one or more access units so that it can be independently decoded. That is, a GOF is a random access unit.

[0337] For example, the bit stream includes a GOF header containing common control information within the GOF.

[0338] For example, a three-dimensional data encoding device is equipped with a processor and memory, and the processor uses the memory to perform the above processing.

[0339] In addition, as described above, the three-dimensional data decoding device performs the processing shown in FIG. 46. The three-dimensional data decoding device decodes three-dimensional data in a time series (e.g., point cloud data of a dynamic object). The three-dimensional data includes position information and attribute information for each time point. The position information and attribute information for the same time point constitute an access unit (AU).

[0340] First, the three-dimensional data decoder decodes position information from the bit stream (S4851). That is, the three-dimensional data decoder generates position information by decoding the encoded position information (encoded position data) included in the bit stream.

[0341] Next, the three-dimensional data decoder decodes attribute information of a processing target from a bit stream by referring to location information at the same time as the attribute information of the processing target (S4852). That is, the three-dimensional data decoder generates attribute information by decoding encoded attribute information (encoded attribute data) included in the bit stream. At this time, the three-dimensional data decoder refers to the decoded location information included in the same access unit as the attribute information.

[0342] According to this, the three-dimensional data decoding device can facilitate the control of references during decoding by using an access unit. Accordingly, the three-dimensional data decoding method can reduce the throughput of the decoding process.

[0343] For example, a three-dimensional data decoder obtains information from a bit stream indicating location information of a reference point of attribute information of a processing target, and decodes the attribute information of the processing target by referring to the location information of the reference point appearing in the obtained information.

[0344] For example, the bit stream includes a position parameter set (position PS) containing control information for position information of each time point, and an attribute parameter set (attribute PS) containing control information for attribute information of each time point. That is, the three-dimensional data decoder decodes the position information of the time point to be processed using the control information included in the position parameter set of the time point to be processed, and decodes the attribute information of the time point to be processed using the control information included in the attribute parameter set of the time point to be processed.

[0345] For example, the bit stream includes a position sequence parameter set (position SPS) containing common control information for position information of multiple times, and an attribute sequence parameter set (attribute SPS) containing common control information for attribute information of multiple times. That is, the three-dimensional data decoder decodes position information of multiple times using control information included in the position sequence parameter set, and decodes attribute information of multiple times using control information included in the attribute sequence parameter set.

[0346] For example, the bit stream includes a stream parameter set (stream PS) containing control information common to position information of multiple times and attribute information of multiple times. That is, the three-dimensional data decoder uses the control information included in the stream parameter set to decode the position information of multiple times and the attribute information of multiple times.

[0347] For example, the bit stream includes an access unit header (AU header) containing common control information within the access unit. That is, the three-dimensional data decoder uses the control information included in the access unit header to decode the position information and attribute information included in the access unit.

[0348] For example, a three-dimensional data decoding device independently decodes a GOF (Group of Frames) consisting of one or more access units. That is, a GOF is a random access unit.

[0349] For example, the bit stream includes a GOF header containing common control information within the GOF. That is, the three-dimensional data decoder uses the control information included in the GOF header to decode position information and attribute information included in the GOF.

[0350] For example, a three-dimensional data decoding device is equipped with a processor and memory, and the processor uses the memory to perform the above processing.

[0351] (Embodiment 5)

[0352] Next, the configuration of the dividing unit (4911) will be described. FIG. 47 is a block diagram of the dividing unit (4911). The dividing unit (4911) includes a slice dividing unit (4931) (Slice Divider), a position information tile dividing unit (4932) (Geometry Tile Divider), and an attribute information tile dividing unit (4933) (Attribute Tile Divider).

[0353] The slice division unit (4931) generates multiple slice position information by dividing position information (Position(Geometry)) into slices. Additionally, the slice division unit (4931) generates multiple slice attribute information by dividing attribute information (Attribute) into slices. Furthermore, the slice division unit (4931) outputs slice additional information (SliceMetaData) including information related to slice division and information generated during slice division.

[0354] The location information tile splitting unit (4932) generates multiple split location information (multiple tile location information) by splitting multiple slice location information into tiles. Additionally, the location information tile splitting unit (4932) outputs location tile additional information (Geometry Tile MetaData) including information related to the tile splitting of location information and information generated in the tile splitting of location information.

[0355] The attribute information tile splitting unit (4933) generates multiple split attribute information (multiple tile attribute information) by splitting multiple slice attribute information into tiles. Additionally, the attribute information tile splitting unit (4933) outputs attribute tile additional information (Attribute Tile MetaData) including information related to the tile splitting of attribute information and information generated in the tile splitting of attribute information.

[0356] Also, the number of slices or tiles to be divided is 1 or more. That is, it is not necessary to divide the slices or tiles.

[0357] Also, here, an example is shown in which tile division is performed after slice division, but slice division may be performed after tile division. In addition, new division types may be defined in addition to slices and tiles, and division may be performed into three or more division types.

[0358] Below, a method for partitioning point cloud data is described. Figure 48 is a diagram showing an example of slice and tile partitioning.

[0359] First, the method of slice division is explained. The division unit (4911) divides the three-dimensional point cloud data into arbitrary point clouds in slice units. In the slice division, the division unit (4911) does not divide the position information and attribute information constituting the point, but divides the position information and attribute information together. That is, the division unit (4911) performs slice division such that the position information and attribute information at any point belong to the same slice. Furthermore, according to these, the number of divisions and the division method can be any method. Also, the minimum unit of division is a point. For example, the number of divisions for position information and attribute information is the same. For example, a three-dimensional point corresponding to the position information after slice division and a three-dimensional point corresponding to the attribute information are included in the same slice.

[0360] Additionally, the dividing unit (4911) generates additional slice information, which is additional information related to the number of divisions and the division method when dividing slices. The additional slice information is identical in location information and attribute information. For example, the additional slice information includes information indicating the reference coordinate position, size, or side length of the bounding box after division. Additionally, the additional slice information includes information indicating the number of divisions and the division type.

[0361] Next, the method of tile division is described. The division unit (4911) divides the slice-divided data into slice location information (G slice) and slice attribute information (A slice), and divides the slice location information and slice attribute information into tile units.

[0362] In addition, Figure 48 shows an example of partitioning into an octal tree structure, and the number of partitions and the partitioning method can be any method.

[0363] Additionally, the dividing unit (4911) may divide location information and attribute information using different dividing methods or using the same dividing method. Additionally, the dividing unit (4911) may divide multiple slices into tiles using different dividing methods or using the same dividing method.

[0364] Additionally, the dividing unit (4911) generates additional tile information related to the number of divisions and the division method when dividing tiles. The additional tile information (location tile additional information and attribute tile additional information) is independent of the location information and attribute information. For example, the additional tile information includes information indicating the reference coordinate position, size, or side length of the bounding box after division. Additionally, the additional tile information includes information indicating the number of divisions and the division type.

[0365] Next, an example of a method for dividing point cloud data into slices or tiles is described. The dividing unit (4911) may use a predetermined method as a method for dividing into slices or tiles, or it may adaptively switch the method used according to the point cloud data.

[0366] When dividing into slices, the dividing unit (4911) divides the three-dimensional space in bulk based on location information and attribute information. For example, the dividing unit (4911) determines the shape of an object and divides the three-dimensional space into slices according to the shape of the object. For example, the dividing unit (4911) extracts objects such as trees or buildings and performs division on an object-by-object basis. For example, the dividing unit (4911) performs slice division so that one or more objects are included in a single slice. Alternatively, the dividing unit (4911) divides a single object into multiple slices.

[0367] In this case, the encoding device may change the encoding method for each slice, for example. For example, the encoding device may use a high-quality compression method for a specific object or a specific part of an object. In this case, the encoding device may store information indicating the encoding method for each slice in additional information (metadata).

[0368] Additionally, the dividing unit (4911) may perform slice division based on map information or location information so that each slice corresponds to a predetermined coordinate space.

[0369] When dividing into tiles, the dividing unit (4911) divides the position information and attribute information independently. For example, the dividing unit (4911) divides the slice into tiles according to the amount of data or throughput. For example, the dividing unit (4911) determines whether the amount of data of the slice (e.g., the number of three-dimensional points included in the slice) is greater than a predetermined threshold. If the amount of data of the slice is greater than the threshold, the dividing unit (4911) divides the slice into tiles. If the amount of data of the slice is less than the threshold, the dividing unit (4911) does not divide the slice into tiles.

[0370] For example, the dividing unit (4911) divides the slice into tiles so that the throughput or processing time in the decoder is within a certain range (less than or equal to a predetermined value). As a result, the throughput per tile in the decoder becomes constant, and distributed processing in the decoder becomes easier.

[0371] Additionally, the division unit (4911) makes the number of divisions of the location information greater than the number of divisions of the attribute information when the processing amounts of the location information and the attribute information are different, for example, when the processing amount of the location information is greater than the processing amount of the attribute information.

[0372] Additionally, for example, depending on the content, in a case where the decoding device can quickly decode and display location information and then slowly decode and display attribute information, the dividing unit (4911) may have the number of divisions for location information greater than the number of divisions for attribute information. As a result, the decoding device can increase the number of parallels for location information, so the processing of location information can be faster than the processing of attribute information.

[0373] In addition, the decoding device does not necessarily need to process sliced ​​or tiled data in parallel, and may determine whether to process them in parallel based on the number or capability of the decoding processing units.

[0374] By dividing in the manner described above, adaptive encoding based on content or objects can be realized. Additionally, parallel processing during decoding can be achieved. As a result, the flexibility of the point cloud encoding system or point cloud decoding system is enhanced.

[0375] FIG. 49 is a diagram illustrating an example of a pattern of slice and tile division. In the diagram, DU is a Data Unit and represents data for a tile or a slice. Additionally, each DU includes a SliceIndex and a TileIndex. In the diagram, the numerical value in the upper right corner of the DU represents the slice index, and the numerical value in the lower left corner of the DU represents the tile index.

[0376] In Pattern 1, regarding slice division, the number of divisions and the division method are the same for G slices and A slices. Regarding tile division, the number of divisions and the division method for G slices are different from the number of divisions and the division method for A slices. Also, the same number of divisions and the division method are used among multiple G slices. The same number of divisions and the division method are used among multiple A slices.

[0377] In Pattern 2, regarding slice division, the number of divisions and the division method are the same for G slices and A slices. Regarding tile division, the number of divisions and the division method for G slices are different from the number of divisions and the division method for A slices. Also, the number of divisions and the division method are different among multiple G slices. The number of divisions and the division method are different among multiple A slices.

[0378] (Embodiment 6)

[0379] The following describes an example of performing slice partitioning after tile partitioning. In autonomous applications such as vehicle self-driving, point cloud data is required not for the entire area, but for the area surrounding the vehicle or the region in the direction of travel. Here, tiles and slices can be used to selectively decode the original point cloud data. By partitioning three-dimensional point cloud data into tiles and further partitioning it into slices, improvements in encoding efficiency or parallel processing can be realized. When partitioning the data, additional information (metadata) is generated, and the generated additional information is sent to a multiplexer.

[0380] FIG. 50 is a block diagram showing the configuration of a first encoding unit (5010) included in a three-dimensional data encoding device according to the present embodiment. The first encoding unit (5010) generates encoded data (encoded stream) by encoding point cloud data using a first encoding method (GPCC (Geometry based PCC)). The first encoding unit (5010) includes a splitting unit (5011), a plurality of position information encoding units (5012), a plurality of attribute information encoding units (5013), an additional information encoding unit (5014), and a multiplexing unit (5015).

[0381] The dividing unit (5011) generates multiple divided data by dividing the point cloud data. Specifically, the dividing unit (5011) generates multiple divided data by dividing the space of the point cloud data into multiple sub-spaces. Here, a sub-space is either a tile or a slice, or a combination of a tile and a slice. More specifically, the point cloud data includes location information, attribute information, and additional information. The dividing unit (5011) divides the location information into multiple divided location information and divides the attribute information into multiple divided attribute information. In addition, the dividing unit (5011) generates additional information regarding the division.

[0382] For example, the dividing unit (5011) first divides the point group into tiles. Next, the dividing unit (5011) further divides the obtained tiles into slices.

[0383] A plurality of location information encoding units (5012) generate a plurality of encoded location information by encoding a plurality of divided location information. For example, the plurality of location information encoding units (5012) process the plurality of divided location information in parallel.

[0384] A plurality of attribute information encoding unit (5013) generates a plurality of encoded attribute information by encoding a plurality of divided attribute information. For example, the plurality of attribute information encoding unit (5013) processes a plurality of divided attribute information in parallel.

[0385] The additional information encoding unit (5014) generates encoded additional information by encoding additional information included in the point cloud data and additional information related to data division generated at the division unit (5011).

[0386] The multiplexer (5015) generates encoded data (encoded stream) by multiplexing a plurality of encoding location information, a plurality of encoding attribute information, and encoding additional information, and transmits the generated encoded data. In addition, the encoding additional information is used during decoding.

[0387] Additionally, FIG. 50 illustrates an example in which the number of location information encoding units (5012) and attribute information encoding units (5013) are each two, but the number of location information encoding units (5012) and attribute information encoding units (5013) may each be one or three or more. Also, multiple divided data may be processed in parallel within the same chip, such as multiple cores within a CPU, processed in parallel by cores of multiple chips, or processed in parallel by multiple cores of multiple chips.

[0388] Next, decoding processing will be described. FIG. 51 is a block diagram showing the configuration of the first decoding unit (5020). The first decoding unit (5020) restores point cloud data by decoding the encoded data (encoded stream) generated by encoding the point cloud data with the first encoding method (GPCC). This first decoding unit (5020) includes a demultiplexer (5021), a plurality of position information decoding units (5022), a plurality of attribute information decoding units (5023), an additional information decoding unit (5024), and a combination unit (5025).

[0389] The demultiplexer (5021) generates a plurality of encoding location information, a plurality of encoding attribute information, and encoding additional information by demultiplexing the encoding data (encoding stream).

[0390] A plurality of location information decoding units (5022) generate a plurality of divided location information by decoding a plurality of encoded location information. For example, the plurality of location information decoding units (5022) process a plurality of encoded location information in parallel.

[0391] A plurality of attribute information decoding unit (5023) generates a plurality of divided attribute information by decoding a plurality of encoded attribute information. For example, the plurality of attribute information decoding unit (5023) processes a plurality of encoded attribute information in parallel.

[0392] A plurality of additional information decoding units (5024) generate additional information by decoding the encoded additional information.

[0393] The combining unit (5025) generates location information by combining a plurality of segmented location information using additional information. The combining unit (5025) generates attribute information by combining a plurality of segmented attribute information using additional information. For example, the combining unit (5025) first generates point cloud data corresponding to a tile by combining decoded point cloud data for a slice using slice additional information. Next, the combining unit (5025) restores the original point cloud data by combining point cloud data corresponding to a tile using tile additional information.

[0394] Additionally, FIG. 50 illustrates an example in which the number of location information decoding units (5022) and attribute information decoding units (5023) are each two, but the number of location information decoding units (5022) and attribute information decoding units (5023) may each be one or three or more. Also, multiple divided data may be processed in parallel within the same chip, such as multiple cores within a CPU, processed in parallel by cores of multiple chips, or processed in parallel by multiple cores of multiple chips.

[0395] Next, we will explain the point cloud data partitioning method. In autonomous applications such as vehicle self-driving, point cloud data is required not for the entire area, but for the area surrounding the vehicle or the region in the direction of travel.

[0396] FIG. 52 is a drawing showing an example of a tile shape. As shown in FIG. 52, various shapes such as circles, rectangles, or ellipses may be used as the tile shape.

[0397] FIG. 53 is a diagram showing examples of tiles and slices. The configuration of the slices may differ from tile to tile. For example, the configuration of the tiles or slices may be optimized based on the amount of data. Alternatively, the configuration of the tiles or slices may be optimized based on the decoding speed.

[0398] In addition, tile division may be performed based on location information. In this case, the attribute information is divided in the same way as the corresponding location information.

[0399] In addition, regarding slice division after tile division, position information and attribute information may be divided into slices by different methods. For example, the method of slice division for each tile may be selected according to the requirements of the application. Depending on the requirements of the application, different slice division methods or tile division methods may be used.

[0400] For example, the dividing unit (5011) divides the point cloud data into one or more tiles based on location information, such as map information, in a two-dimensional shape viewed from above. Then, the dividing unit (5011) divides each tile into one or more slices.

[0401] Additionally, the dividing section (5011) may divide the position information (Geometry) and attribute information (Attribute) into slices in the same way.

[0402] In addition, location information and attribute information may each be of one type or two or more types. Also, in the case of point cloud data that does not have attribute information, attribute information may be omitted.

[0403] FIG. 54 is a block diagram of a dividing unit (5011). The dividing unit (5011) includes a tile dividing unit (5031) (Tile Divider), a position information slice dividing unit (5032) (Geometry Slice Divider), and an attribute information slice dividing unit (5033) (Attribute Slice Divider).

[0404] The tile splitting unit (5031) generates multiple tile position information by dividing position information (Position(Geometry)) into tiles. Additionally, the tile splitting unit (5031) generates multiple tile attribute information by dividing attribute information (Attribute) into tiles. Additionally, the tile splitting unit (5031) outputs tile additional information (TileMetaData) including information related to tile splitting and information generated in tile splitting.

[0405] The location information slice division unit (5032) generates multiple divided location information (multiple slice location information) by dividing multiple tile location information into slices. Additionally, the location information slice division unit (5032) outputs location slice additional information (Geometry Slice MetaData) including information related to the slice division of location information and information generated in the slice division of location information.

[0406] The attribute information slice division unit (5033) generates multiple divided attribute information (multiple slice attribute information) by dividing multiple tile attribute information into slices. Additionally, the attribute information slice division unit (5033) outputs attribute slice additional information (Attribute Slice MetaData) including information related to the slice division of attribute information and information generated in the slice division of attribute information.

[0407] Next, examples of tile shapes will be described. The entire three-dimensional map is divided into multiple tiles. The data of the multiple tiles is optionally transmitted to a three-dimensional data decoder. Alternatively, the data of the multiple tiles is transmitted to the three-dimensional data decoder in order, starting with the data of highest importance. Depending on the situation, the tile shape may be selected from multiple shapes.

[0408] FIG. 55 is a diagram showing an example of a map viewed from above using point cloud data obtained from LiDAR. The example shown in FIG. 55 is point cloud data of a highway and includes a flyover section.

[0409] FIG. 56 is a diagram showing an example of dividing the point cloud data shown in FIG. 55 into square tiles. Such square division can be easily performed on a map server. Also, for normal roads, the height of the tiles is set low. For grade-separated intersections, the height of the tiles is set higher than that of normal roads so that the tiles include the grade-separated intersections.

[0410] FIG. 57 is a diagram showing an example in which the point cloud data shown in FIG. 55 is divided into circular tiles. In this case, adjacent tiles may overlap when viewed from a plane. When a vehicle requires point cloud data of a surrounding area, the three-dimensional data encoding device transmits point cloud data of the area of ​​a circular column (a circle when viewed from the top) surrounding the vehicle to the vehicle.

[0411] Also, as in the example of Fig. 56, the height of the tile is set low for normal roads. In the grade-separated intersection, the height of the tile is set higher than that of normal roads so that the tile covers the grade-separated intersection.

[0412] A three-dimensional data encoding device may change the height of a tile according to, for example, the shape or height of a road or building. Additionally, a three-dimensional data encoding device may change the height of a tile according to location information or area information. Furthermore, a three-dimensional data encoding device may change the height of a tile for each tile. Alternatively, a three-dimensional data encoding device may change the height of a tile for each section containing multiple tiles. That is, a three-dimensional data encoding device may make the heights of multiple tiles within a section identical. Additionally, tiles of different heights may overlap when viewed from the top surface.

[0413] FIG. 58 is a drawing showing an example of tile division using tiles of various shapes, sizes, or heights. The shape of the tiles can be any shape, any size, or a combination thereof.

[0414] For example, in addition to the example of dividing into square tiles without overlap as described above and the example of dividing into circular tiles with overlap, the three-dimensional data encoding device may also perform division into square tiles with overlap. Also, the shape of the tile may not be square or circular, may use a polygon with three or more vertices, or may use a shape without vertices.

[0415] Also, the shape of the tile may be of two or more types, and tiles of different shapes may overlap. Also, there may be one or more types of tile shapes, and within the same shape being divided, shapes of different sizes may be combined, and they may overlap.

[0416] For example, in areas without objects such as roads, tiles larger than those in areas where objects exist are used. Additionally, the three-dimensional data encoding device may adaptively change the shape or size of the tiles depending on the object.

[0417] In addition, for example, since it is highly likely that a three-dimensional data encoding device will need to read tiles far ahead of the vehicle, which is the direction of travel, the tiles in the direction of travel can be set to a large size, and since it is unlikely that the vehicle will move to the side, the tiles in the side can be set to a smaller size than the tiles in the direction of travel.

[0418] FIG. 59 is a diagram illustrating an example of tile data stored on a server. For example, point cloud data is pre-tile-divided and encoded, and the obtained encoded data is stored on a server. A user obtains the desired tile data from the server when needed. Alternatively, the server (three-dimensional data encoding device) may perform tile division and encoding to include the data desired by the user according to the user's instructions.

[0419] For example, if the moving speed of a moving object (vehicle) is fast, it can be considered that more extensive point cloud data is required. Accordingly, the server may determine the shape and size of the tiles based on the estimated speed of the vehicle (e.g., the legal speed limit of the road, the speed of the vehicle that can be estimated from the width and shape of the road, or the statistical speed, etc.) and perform tile division. Alternatively, as shown in FIG. 59, the server may encode multiple tiles of shapes or sizes in advance and store the obtained data. The moving object may acquire data of tiles of appropriate shapes and sizes according to the direction and speed of the moving object.

[0420] FIG. 60 is a diagram illustrating an example of a system for tile division. As shown in FIG. 60, the shape and area of ​​the tile may be determined based on the location of an antenna (base station) which is a communication means for transmitting point cloud data, or the communication area supported by the antenna. Alternatively, when point cloud data is generated by a sensor such as a camera, the shape and area of ​​the tile may be determined based on the location of the sensor or the target range (detection range) of the sensor.

[0421] One tile may be assigned to one antenna or sensor, or one tile may be assigned to multiple antennas or sensors. Multiple tiles may be assigned to one antenna or sensor. An antenna or sensor may be fixed or movable.

[0422] For example, the coded data divided into tiles may be managed by a server connected to an antenna or sensor for the area assigned to the tile. The server may manage the coded data of its own area and the tile information of an adjacent area. Multiple coded data of multiple tiles may be managed in a centralized management server (cloud) that manages multiple servers corresponding to each tile. Alternatively, the antenna or sensor may be directly connected to the centralized management server without installing a server corresponding to the tile.

[0423] In addition, the target range of the antenna or sensor may vary depending on the power of the radio waves, differences in the device, and installation conditions, and the shape and size of the tile may also change accordingly. Depending on the target range of the antenna or sensor, a slice may be assigned instead of a tile, or a PCC frame may be assigned.

[0424] Next, we will explain the method of dividing tiles into slices. Encoding efficiency can be improved by assigning similar objects to the same slice.

[0425] For example, a three-dimensional data encoding device may recognize objects (roads, buildings, trees, etc.) using features of point cloud data and perform slice partitioning by clustering point clouds for each object.

[0426] Alternatively, the three-dimensional data encoding device may perform slice partitioning by grouping objects having the same attributes and assigning slices to each group. Here, attributes refer to information regarding movement, for example, and grouping is performed by classifying objects into dynamic information such as pedestrians and vehicles, quasi-dynamic information such as accidents and traffic congestion, quasi-static information such as traffic regulations and road construction, and static information such as road surfaces and structures.

[0427] In addition, data may be duplicated across multiple slices. For example, when slicing for multiple object groups, any object may belong to one object group or to two or more object groups.

[0428] FIG. 61 is a drawing showing an example of this slice division. For example, in the example shown in FIG. 61, the tile is a rectangular prism. In addition, the tile may be in the shape of a cylinder or any other shape.

[0429] The point clouds contained in the tile are grouped into object groups, such as roads, buildings, and trees. Then, slicing is performed so that each object group is contained in a single slice. Subsequently, each slice is encoded individually.

[0430] Next, a method for encoding divided data will be described. A three-dimensional data encoding device (a first encoding unit (5010)) encodes each of the divided data. When encoding attribute information, the three-dimensional data encoding device generates dependency relationship information as additional information, which indicates which configuration information (location information, additional information, or other attribute information) was used for encoding. That is, the dependency relationship information represents, for example, the configuration information of a reference location (dependency location). In this case, the three-dimensional data encoding device generates dependency relationship information based on configuration information corresponding to the divided shape of the attribute information. Additionally, the three-dimensional data encoding device may generate dependency relationship information based on configuration information corresponding to multiple divided shapes.

[0431] Dependency relationship information may be generated by the three-dimensional data encoding device and transmitted to the three-dimensional data decoder. Alternatively, the three-dimensional data decoder may generate dependency relationship information, and the three-dimensional data encoding device may not transmit the dependency relationship information. Furthermore, the dependency relationships used by the three-dimensional data encoding device may be predetermined, and the three-dimensional data encoding device may not transmit the dependency relationship information.

[0432] FIG. 62 is a diagram illustrating an example of the dependency relationship of each data. In the diagram, the tip of the arrow indicates the dependency, and the root of the arrow indicates the source of dependency. The three-dimensional data decoding device decodes the data in the order from the dependency to the source of dependency. Also, the data indicated by solid lines in the diagram is the data that is actually transmitted, and the data indicated by dotted lines is the data that is not transmitted.

[0433] Also, in the same drawing, G represents location information and A represents attribute information. Gt1 represents location information of tile number 1, and Gt2 represents location information of tile number 2. Gt1s1 represents location information of tile number 1 and also slice number 1, Gt1s2 represents location information of tile number 1 and also slice number 2, Gt2s1 represents location information of tile number 2 and also slice number 1, and Gt2s2 represents location information of tile number 2 and also slice number 2. Likewise, At1 represents attribute information of tile number 1, and At2 represents attribute information of tile number 2. At1s1 represents the attribute information of tile number 1 and also slice number 1, At1s2 represents the attribute information of tile number 1 and also slice number 2, At2s1 represents the attribute information of tile number 2 and also slice number 1, and At2s2 represents the attribute information of tile number 2 and also slice number 2.

[0434] Mtile represents tile additional information, MGslice represents location slice additional information, and MAslice represents attribute slice additional information. Dt1s1 represents dependency relationship information of attribute information At1s1, and Dt2s1 represents dependency relationship information of attribute information At2s1.

[0435] In addition, depending on the application, different tile division or slice division structures may be used.

[0436] Additionally, the three-dimensional data encoding device may rearrange the data in the order of decoding so that there is no need to rearrange the data in the three-dimensional data decoding device. Furthermore, the data may be rearranged in the three-dimensional data decoding device, or the data may be rearranged in both the three-dimensional data encoding device and the three-dimensional data decoding device.

[0437] FIG. 63 is a diagram illustrating an example of the decoding order of data. In the example of FIG. 63, decoding is performed in order starting from the data on the left. Among data in a dependency relationship, the three-dimensional data decoding device decodes the dependent data first. For example, the three-dimensional data encoding device pre-arranges the data to achieve this order and transmits it. Additionally, any order is acceptable as long as the dependent data takes precedence. Furthermore, the three-dimensional data encoding device may transmit additional information and dependency relationship information before the data.

[0438] Additionally, the three-dimensional data decoding device may selectively decode tiles based on a request from an application and information obtained from the NAL unit header. FIG. 64 is a diagram showing an example of encoded tile data. For example, the decoding order of the tiles is arbitrary. That is, there may be no dependency relationship between the tiles.

[0439] Next, the configuration of the combination unit (5025) included in the first decoding unit (5020) will be explained. FIG. 65 is a block diagram showing the configuration of the combination unit (5025). The combination unit (5025) includes a position information slice combination unit (5041) (Geometry Slice Combiner), an attribute information slice combination unit (5042) (Attribute Slice Combiner), and a tile combination unit (Tile Combiner).

[0440] The location information slice combining unit (5041) generates multiple tile location information by combining multiple divided location information using location slice additional information. The attribute information slice combining unit (5042) generates multiple tile attribute information by combining multiple divided attribute information using attribute slice additional information.

[0441] The tile combining unit (5043) generates location information by combining multiple tile location information using tile additional information. Additionally, the tile combining unit (5043) generates attribute information by combining multiple tile attribute information using tile additional information.

[0442] In addition, the number of slices or tiles to be divided is 1 or more. That is, the division of slices or tiles does not need to be performed.

[0443] Next, the composition of slice-divided or tile-divided encoded data and the method of storing the encoded data in a NAL unit (multiplexing method) will be described. FIG. 66 is a diagram showing the composition of encoded data and the method of storing the encoded data in a NAL unit.

[0444] Encoded data (segmentation location information and segmentation attribute information) is stored in the payload of the NAL unit.

[0445] Encoded data includes a header and a payload. The header includes identification information for identifying the data included in the payload. This identification information includes, for example, the type of slice division or tile division (slice_type, tile_type), index information for identifying the slice or tile (slice_idx, tile_idx), location information of the data (slice or tile), or the address of the data. Index information for identifying the slice is also referred to as SliceIndex. Index information for identifying the tile is also referred to as TileIndex. Furthermore, the type of division is, for example, a method based on the object shape as described above, a method based on map information or location information, or a method based on the amount of data or throughput.

[0446] Additionally, the header of the encoded data includes identification information indicating dependency relationships. That is, if there is a dependency relationship between data, the header includes identification information for referencing the dependency from the dependency source. For example, the header of the data of the dependency includes identification information for specifying the data. The header of the data of the dependency includes identification information indicating the dependency. Furthermore, if the identification information for specifying data, additional information related to slice partitioning or tile partitioning, and identification information indicating dependency relationships can be identified or derived from other information, these details may be omitted.

[0447] Next, the flow of encoding and decoding processing of point cloud data according to the present embodiment will be described. FIG. 67 is a flowchart of encoding processing of point cloud data according to the present embodiment.

[0448] First, the three-dimensional data encoding device determines the division method to be used (S5011). This division method includes whether to perform tile division or slice division. In addition, the division method may include the number of divisions when performing tile division or slice division, and the type of division. The type of division is a method based on object shape as described above, a method based on map information or location information, or a method based on data amount or throughput. In addition, the division method may be predetermined.

[0449] When tile division is performed (Yes in S5012), the three-dimensional data encoding device generates multiple tile location information and multiple tile attribute information by dividing the location information and attribute information in batches (S5013). Additionally, the three-dimensional data encoding device generates tile additional information related to tile division. Furthermore, the three-dimensional data encoding device may divide the location information and attribute information independently.

[0450] When slice division is performed (Yes in S5014), the three-dimensional data encoding device generates multiple divided location information and multiple divided attribute information by independently dividing multiple tile location information and multiple tile attribute information (or location information and attribute information) (S5015). Additionally, the three-dimensional data encoding device generates location slice additional information and attribute slice additional information related to the slice division. Furthermore, the three-dimensional data encoding device may divide the tile location information and tile attribute information in a batch.

[0451] Next, the three-dimensional data encoding device generates multiple encoded location information and multiple encoded attribute information by encoding each of the multiple segmented location information and multiple segmented attribute information (S5016). In addition, the three-dimensional data encoding device generates dependency relationship information.

[0452] Next, the three-dimensional data encoding device generates encoded data (encoded stream) by unitizing (multiplexing) a plurality of encoding position information, a plurality of encoding attribute information and additional information into NAL units (S5017). In addition, the three-dimensional data encoding device transmits the generated encoded data.

[0453] FIG. 68 is a flowchart of the decoding process of point cloud data according to the present embodiment. First, the three-dimensional data decoding device determines a division method by interpreting additional information related to the division method (tile additional information, position slice additional information, and attribute slice additional information) included in the encoded data (encoded stream) (S5021). This division method includes whether to perform tile division or slice division. In addition, the division method may include the number of divisions when performing tile division or slice division, and the type of division, etc.

[0454] Next, the three-dimensional data decoding device generates segmentation position information and segmentation attribute information by decoding a plurality of encoding position information and a plurality of encoding attribute information included in the encoded data using dependency relationship information included in the encoded data (S5022).

[0455] When it is indicated that slice division is performed based on additional information (Yes in S5023), the three-dimensional data decoder generates multiple tile location information and multiple tile attribute information by combining multiple division location information and multiple division attribute information in each manner based on the location slice additional information and the attribute slice additional information (S5024). Additionally, the three-dimensional data decoder may combine multiple division location information and multiple division attribute information in the same manner.

[0456] When it is indicated that tile division is performed based on additional information (Yes in S5025), the three-dimensional data decoding device generates position information and attribute information by combining a plurality of tile position information and a plurality of tile attribute information (a plurality of division position information and a plurality of division attribute information) in the same way based on the tile additional information (S5026). Additionally, the three-dimensional data decoding device may combine the plurality of tile position information and the plurality of tile attribute information in different ways.

[0457] Next, tile supplementary information is described. The three-dimensional data encoding device generates tile supplementary information, which is metadata regarding the tile division method, and transmits the generated tile supplementary information to the three-dimensional data decoding device.

[0458] FIG. 69 is a diagram showing an example of the syntax of tile metadata. As shown in FIG. 69, for example, tile metadata includes type_of_divide information, topview_shape information, a tile_overlap_flag, type_of_overlap information, height information, tile_number, and tile position information (global_position, relative_position).

[0459] The type_of_divide information indicates the method of dividing the tile. For example, the type_of_divide information indicates whether the method of dividing the tile is based on map information, that is, top_view, or other.

[0460] Shape information (topview_shape) is included in the tile supplementary information, for example, when the method of dividing the tile is based on the top view. Shape information represents the shape of the tile as seen from the top view. For example, this shape includes squares and circles. In addition, this shape may include ellipses, rectangles, or polygons other than squares, and may include other shapes. Furthermore, shape information is not limited to the shape of the tile as seen from the top view, and may represent the three-dimensional shape of the tile (e.g., cubes and cylinders).

[0461] The tile_overlap_flag indicates whether tiles overlap. For example, the overlap flag is included in the tile supplementary information when the tile partitioning method is based on the top view. In this case, the overlap flag indicates whether tiles overlap when viewed from the top view. Additionally, the overlap flag may indicate whether tiles overlap in three-dimensional space.

[0462] Overlap information (type_of_overlap) is included in the tile supplementary information, for example, when tiles overlap. Overlap information indicates the method of tile overlap, etc. For example, overlap information indicates the size of the overlapping area, etc.

[0463] The height information (tile_height) indicates the height of the tile. Additionally, the height information may include information indicating the shape of the tile. For example, if the shape of the tile when viewed from the top surface is a rectangle, the information may indicate the lengths of the sides (height and width) of the rectangle. Furthermore, if the shape of the tile when viewed from the top surface is a circle, the information may indicate the diameter or radius of the circle.

[0464] Additionally, the height information may represent the height of each individual tile or a common height across multiple tiles. Furthermore, multiple height types, such as roads and overpasses, may be pre-set, and the height of each height type and the height type of each tile may be displayed based on the height information. Alternatively, the height of each height type may be pre-defined, and the height type of each tile may be displayed based on the height information. In other words, the height of each height type does not need to be displayed in the height information.

[0465] Tile_number indicates the number of tiles. Additionally, tile supplementary information may include information indicating the spacing between tiles.

[0466] Tile position information (global_position, relative_position) is information used to specify the location of each tile. For example, tile position information represents the absolute or relative coordinates of each tile.

[0467] In addition, some or all of the above information may be formed for each tile, or for each of multiple tiles (e.g., for each frame or for each of multiple frames).

[0468] The three-dimensional data encoding device may include tile supplementary information in SEI (Supplemental Enhancement Information) and transmit it. Alternatively, the three-dimensional data encoding device may store tile supplementary information in an existing parameter set (PPS, GPS, or APS, etc.) and transmit it.

[0469] For example, if tile additional information changes per frame, the tile additional information may be stored in the parameter set for each frame (e.g., GPS or APS). If the tile additional information does not change within a sequence, the tile additional information may be stored in the parameter set for each sequence (e.g., Position SPS or Attribute SPS). Furthermore, if the same tile splitting information is used in both the position information and the attribute information, the tile additional information may be stored in the parameter set of the PCC stream (Stream PS).

[0470] Additionally, the tile additional information may be stored in any one of the above parameter sets or in multiple parameter sets. Additionally, the tile additional information may be stored in the header of the encoded data. Additionally, the tile additional information may be stored in the header of the NAL unit.

[0471] In addition, all or part of the tile additional information may be stored in either the header of the segmentation location information or the header of the segmentation attribute information, and not in the other. For example, if the same tile additional information is used in the location information and the attribute information, the tile additional information may be included in the header of either the location information or the attribute information. For example, if the attribute information depends on the location information, the location information is processed first. Therefore, the tile additional information may be included in the header of the location information, and the tile additional information may not be included in the header of the attribute information. In this case, the three-dimensional data decoding device determines, for example, that the attribute information of the dependent source belongs to the same tile as the tile of the dependent location information.

[0472] The three-dimensional data decoding device reconstructs tile-divided point cloud data based on tile additional information. If there is duplicate point cloud data, the three-dimensional data decoding device identifies multiple duplicate point cloud data, selects one of them, or merges multiple point cloud data.

[0473] Additionally, the three-dimensional data decoding device may perform decoding using tile supplementary information. For example, if multiple tiles overlap, the three-dimensional data decoding device may perform decoding for each tile and generate point cloud data by performing processing (e.g., smoothing or filtering) using the multiple decoded data. This makes it possible to achieve high-precision decoding.

[0474] FIG. 70 is a diagram showing an example of a configuration of a system including a three-dimensional data encoding device and a three-dimensional data decoding device. A tile splitting unit (5051) splits point cloud data including location information and attribute information into a first tile and a second tile. Also, the tile splitting unit (5051) sends tile additional information related to tile splitting to a decoding unit (5053) and a tile combining unit (5054).

[0475] The encoding unit (5052) generates encoded data by encoding the first tile and the second tile.

[0476] The decoding unit (5053) restores the first tile and the second tile by decoding the encoded data generated by the encoding unit (5052). The tile combining unit (5054) restores the point cloud data (location information and attribute information) by combining the first tile and the second tile using tile additional information.

[0477] Next, slice supplementary information is described. The three-dimensional data encoding device generates slice supplementary information, which is metadata regarding the slice division method, and transmits the generated slice supplementary information to the three-dimensional data decoding device.

[0478] FIG. 71 is a diagram showing an example of the syntax of slice additional information (SliceMetaData). As shown in FIG. 71, for example, the slice additional information includes division method information (type_of_divide), overlap flag (slice_overlap_flag), overlap information (type_of_overlap), number of slices (slice_number), slice position information (global_position, relative_position), and slice size information (slice_bounding_box_size).

[0479] The division method information (type_of_divide) indicates the division method of the slice. For example, the division method information indicates whether the division method of the slice is an object based on the object information as shown in FIG. 61. Additionally, the slice additional information may include information indicating the method of dividing the object. For example, this information indicates whether to divide one object into multiple slices or to assign it to one slice. Also, this information may indicate the number of divisions when dividing one object into multiple slices.

[0480] The overlap flag (slice_overlap_flag) indicates whether slices overlap. The overlap information (type_of_overlap) is included in the slice supplementary information, for example, if slices overlap. The overlap information indicates the method of slice overlap, etc. For example, the overlap information indicates the size of the overlapping area, etc.

[0481] The slice_number indicates the number of slices.

[0482] Slice position information (global_position, relative_position) and slice size information (slice_bounding_box_size) are information regarding the area of ​​a slice. Slice position information is information used to specify the location of each slice. For example, slice position information indicates the absolute or relative coordinates of each slice. Slice size information (slice_bounding_box_size) indicates the size of each slice. For example, slice size information indicates the size of the bounding box of each slice.

[0483] The three-dimensional data encoding device may include slice additional information in the SEI and transmit it. Alternatively, the three-dimensional data encoding device may store the slice additional information in an existing parameter set (PPS, GPS, or APS, etc.) and transmit it.

[0484] For example, if slice additional information changes per frame, it may be stored in the parameter set for each frame (e.g., GPS or APS). If slice additional information does not change within a sequence, it may be stored in the parameter set for each sequence (e.g., Position SPS or Attribute SPS). Additionally, if the same slice partitioning information is used in both position information and attribute information, it may be stored in the parameter set of the PCC stream (Stream PS).

[0485] Additionally, the slice additional information may be stored in any one of the above parameter sets or in multiple parameter sets. Additionally, the slice additional information may be stored in the header of the encoded data. Additionally, the slice additional information may be stored in the header of the NAL unit.

[0486] In addition, all or part of the slice additional information may be stored in either the header of the split location information or the header of the split attribute information, and may not be stored in the other. For example, if the same slice additional information is used in the location information and the attribute information, the slice additional information may be included in the header of either the location information or the attribute information. For example, if the attribute information depends on the location information, the location information is processed first. Therefore, the slice additional information may be included in the header of the location information, and the slice additional information may not be included in the header of the attribute information. In this case, the three-dimensional data decoder determines, for example, that the attribute information of the dependent source belongs to the same slice as the slice of the location information of the dependent source.

[0487] The three-dimensional data decoder reconstructs sliced ​​point cloud data based on slice addition information. If there is duplicate point cloud data, the three-dimensional data decoder identifies multiple duplicate point cloud data, selects one of them, or merges multiple point cloud data.

[0488] Additionally, the three-dimensional data decoder may perform decoding using slice supplementary information. For example, if multiple slices overlap, the three-dimensional data decoder may perform decoding for each slice and generate point cloud data by performing processing (e.g., smoothing or filtering) using the multiple decoded data. This makes it possible to achieve high-precision decoding.

[0489] FIG. 72 is a flowchart of a three-dimensional data encoding process including the generation of tile additional information by a three-dimensional data encoding device according to the present embodiment.

[0490] First, the three-dimensional data encoding device determines a method for dividing the tile (S5031). Specifically, the three-dimensional data encoding device determines whether to use a top-view division method or an other method as a method for dividing the tile. In addition, the three-dimensional data encoding device determines the shape of the tile when using a top-view division method. In addition, the three-dimensional data encoding device determines whether the tile overlaps with other tiles.

[0491] If the tile division method determined in step S5031 is a division method based on top view (Yes in S5032), the three-dimensional data encoding device records in the tile additional information that the tile division method is a division method based on top view (top_view) (S5033).

[0492] Meanwhile, if the tile division method determined in step S5031 is other than the division method based on the top view (No in S5032), the three-dimensional data encoding device records in the tile additional information that the tile division method is a method other than the division method based on the top view (top_view) (S5034).

[0493] Additionally, if the shape of the tile viewed from the top surface determined in step S5031 is a square (square in S5035), the three-dimensional data encoding device records that the shape of the tile viewed from the top surface is a square in the tile additional information (S5036). Meanwhile, if the shape of the tile viewed from the top surface determined in step S5031 is a circle (circle in S5035), the three-dimensional data encoding device records that the shape of the tile viewed from the top surface is a circle in the tile additional information (S5037).

[0494] Next, the three-dimensional data encoding device determines whether the tile overlaps with another tile (S5038). If the tile overlaps with another tile (Yes in S5038), the three-dimensional data encoding device records that the tile overlaps in the tile supplementary information (S5039). Meanwhile, if the tile does not overlap with another tile (No in S5038), the three-dimensional data encoding device records that the tile does not overlap in the tile supplementary information (S5040).

[0495] Next, the three-dimensional data encoding device divides the tiles based on the tile division method determined in step S5031, encodes each tile, and transmits the generated encoded data and tile additional information (S5041).

[0496] FIG. 73 is a flowchart of three-dimensional data decoding processing using tile additional information by a three-dimensional data decoding device according to the present embodiment.

[0497] First, the three-dimensional data decoder interprets tile additional information included in the bit stream (S5051).

[0498] If the additional tile information indicates that a tile does not overlap with other tiles (No in S5052), the three-dimensional data decoder generates point cloud data for each tile by decoding each tile (S5053). Next, the three-dimensional data decoder reconstructs point cloud data from the point cloud data of each tile based on the tile division method and the shape of the tile indicated by the additional tile information (S5054).

[0499] Meanwhile, if the additional tile information indicates that a tile overlaps with another tile (Yes in S5052), the three-dimensional data decoder generates point cloud data for each tile by decoding each tile. Additionally, the three-dimensional data decoder identifies the overlapping portion of the tile based on the additional tile information (S5055). Furthermore, the three-dimensional data decoder may perform decoding processing for the overlapping portion using multiple overlapping information. Next, the three-dimensional data decoder reconstructs point cloud data from the point cloud data of each tile based on the tile division method, tile shape, and overlap information indicated by the additional tile information (S5056).

[0500] Variations regarding slices, etc., are described below. A three-dimensional data encoding device may transmit information indicating the type of object (road, building, tree, etc.) or attribute (dynamic information, static information, etc.) as supplementary information. Alternatively, encoding parameters may be predetermined according to the object, and the three-dimensional data encoding device may notify the three-dimensional data decoding device of the encoding parameters by transmitting the type or attribute of the object.

[0501] Regarding the encoding and transmission order of slice data, the following methods may be used. For example, the three-dimensional data encoding device may encode slice data in order starting from data that is easy to recognize or cluster. Alternatively, the three-dimensional data encoding device may perform encoding in order starting from slice data that has finished clustering quickly. Furthermore, the three-dimensional data encoding device may transmit the encoded slice data in order. Alternatively, the three-dimensional data encoding device may transmit slice data in order of high decoding priority in the application. For example, if dynamic information has a high decoding priority, the three-dimensional data encoding device may transmit slice data in order starting from slices grouped by dynamic information.

[0502] In addition, if the sequence number of the encoded data and the sequence number of the decoding priority are different, the three-dimensional data encoding device may transmit the encoded data after rearranging it. In addition, when accumulating encoded data, the three-dimensional data encoding device may accumulate the encoded data after rearranging it.

[0503] The application (three-dimensional data decoder) requests the server (three-dimensional data encoder) to transmit a slice containing the desired data. The server transmits the slice data required by the application and does not need to transmit unnecessary slice data.

[0504] The application requests the server to send tiles containing the desired data. The server sends the tile data required by the application and does not need to send unnecessary tile data.

[0505] As described above, the three-dimensional data encoding device according to the present embodiment performs the processing shown in FIG. 74. First, the three-dimensional data encoding device generates a plurality of encoded data by encoding a plurality of sub-spaces (e.g., tiles) that divide a target space containing a plurality of three-dimensional points (S5061). The three-dimensional data encoding device generates a bit stream including the plurality of encoded data and first information (e.g., topview_shape) representing the shape of the plurality of sub-spaces (S5062).

[0506] According to this, the three-dimensional data encoding device can select any shape from multiple types of sub-space shapes, thereby improving encoding efficiency.

[0507] For example, the above shape is the shape of the plurality of subspaces in two dimensions or in three dimensions. For example, the above shape is the shape of the plurality of subspaces viewed from above. That is, the first information represents the shape of the subspace viewed from a specific direction (e.g., upward). In other words, the first information represents the shape of the subspace viewed from below. For example, the above shape is a rectangle or a circle.

[0508] For example, the bit stream includes second information (e.g., tile_overlap_flag) indicating whether the plurality of sub-sections overlap.

[0509] According to this, the three-dimensional data encoding device can duplicate subspaces, so it can generate subspaces without complicating the shape of the subspaces.

[0510] For example, the bit stream includes third information (e.g., type_of_divide) indicating whether the method of dividing the plurality of sub-sections is a method of dividing using a face view.

[0511] For example, the bit stream includes a fourth piece of information (e.g., tile_height) indicating at least one of the height, width, depth, and radius of the plurality of sub-sections.

[0512] For example, the bit stream includes fifth information (e.g., global_position or relative_position) indicating the position of each of the plurality of sub-sections.

[0513] For example, the bit stream includes sixth information (e.g., tile_number) indicating the number of the plurality of sub-sections.

[0514] For example, the bit stream includes seventh information indicating the interval of the plurality of sub-intervals.

[0515] For example, a three-dimensional data encoding device is equipped with a processor and memory, and the processor uses the memory to perform the above processing.

[0516] In addition, the three-dimensional data decoding device according to the present embodiment performs the processing shown in FIG. 75. First, the three-dimensional data decoding device restores the plurality of subspaces by decoding the plurality of encoded data generated by dividing the target space containing the plurality of three-dimensional points included in the bit stream into a plurality of subspaces (e.g., tiles) (S5071). The three-dimensional data decoding device restores the target space by combining the plurality of subspaces using first information (e.g., topview_shape) representing the shape of the plurality of subspaces included in the bit stream (S5072). For example, the three-dimensional data decoding device can determine the location and range of each subspace within the target space by recognizing the shape of the plurality of subspaces using the first information. The three-dimensional data decoding device can combine the plurality of subspaces based on the location and range of the identified plurality of subspaces. As a result, the three-dimensional data decoding device can correctly combine the plurality of subspaces.

[0517] For example, the above shape is a shape in two dimensions or a shape in three dimensions of the plurality of subspaces. For example, the above shape is a rectangle or a circle.

[0518] For example, the bit stream includes second information (e.g., tile_overlap_flag) indicating whether the plurality of sub-sections overlap. In the reconstruction of the target space, the three-dimensional data decoder combines the plurality of sub-spaces by additionally using the second information. For example, the three-dimensional data decoder uses the second information to determine whether the sub-spaces overlap. If the sub-spaces overlap, the three-dimensional data decoder identifies the overlapping region and performs a predetermined response for the identified overlapping region.

[0519] For example, the bit stream includes third information (e.g., type_of_divide) indicating whether the division method of the plurality of sub-sections is a division method using a top view. If the third information indicates that the division method of the plurality of sub-sections is a division method using a top view, the three-dimensional data decoding device combines the plurality of sub-spaces using the first information.

[0520] For example, the bit stream includes fourth information (e.g., tile_height) representing at least one of the height, width, depth, and radius of the plurality of sub-sections. In the reconstruction of the target space, the three-dimensional data decoding device further uses the fourth information to combine the plurality of sub-spaces. For example, the three-dimensional data decoding device can determine the location and range of each sub-space within the target space by recognizing the height of the plurality of sub-spaces using the fourth information. The three-dimensional data decoding device can combine the plurality of sub-spaces based on the determined location and range of the plurality of sub-spaces.

[0521] For example, the bit stream includes fifth information (e.g., global_position or relative_position) indicating the position of each of the plurality of sub-sections. In the reconstruction of the target space, the three-dimensional data decoder combines the plurality of sub-spaces by additionally using the fifth information. For example, the three-dimensional data decoder can determine the position of each sub-space within the target space by recognizing the positions of the plurality of sub-spaces using the fifth information. The three-dimensional data decoder can combine the plurality of sub-spaces based on the identified positions of the plurality of sub-spaces.

[0522] For example, the bit stream includes sixth information (e.g., tile_number) indicating the number of the plurality of sub-sections. In the reconstruction of the target space, the three-dimensional data decoding device further uses the sixth information to combine the plurality of sub-spaces.

[0523] For example, the bit stream includes seventh information indicating the interval of the plurality of sub-sections. In the restoration of the target space, the three-dimensional data decoding device further uses the seventh information to combine the plurality of sub-spaces. For example, the three-dimensional data decoding device can determine the location and range of each sub-space within the target space by recognizing the interval of the plurality of sub-spaces using the seventh information. The three-dimensional data decoding device can combine the plurality of sub-spaces based on the determined locations and ranges of the plurality of sub-spaces.

[0524] For example, a three-dimensional data decoding device is equipped with a processor and memory, and the processor uses the memory to perform the above processing.

[0525] (Embodiment 7)

[0526] In order to divide point cloud data into tiles and slices and efficiently encode or decode the divided data, it is necessary to control them appropriately on the encoding side and the decoding side. By encoding and decoding the divided data independently without dependencies between the divided data, the divided data can be processed in parallel on each thread / core using a multi-threaded or multi-core processor, thereby improving performance.

[0527] There are various methods for dividing point cloud data into tiles and slices, for example, there are methods for dividing based on attributes of objects in the point cloud data such as road surfaces or characteristics such as color information such as the green color of the point cloud data.

[0528] CABAC stands for Context-Based Adaptive Binary Arithmetic Coding, and it is a coding method that realizes high compression rate arithmetic coding (entropy coding) by increasing the precision of the probability by sequentially updating the context (a model that estimates the probability of occurrence of input binary symbols) based on the coding completion information.

[0529] To process partitioned data, such as tiles or slices, in parallel, it is necessary to be able to encode or decode each partitioned data independently. However, to perform CABAC independently between partitioned data, it is necessary to initialize CABAC at the beginning of the partitioned data during encoding and decoding, but there is no structure for this.

[0530] The CABACABAC initialization flag is used to initialize the CABAC during CABAC encoding and decoding.

[0531] FIG. 76 is a flowchart showing the initialization process of CABACCABAC according to the CABAC initialization flag in encoding or decoding.

[0532] The three-dimensional data encoding device or the three-dimensional data decoding device determines whether the CABAC initialization flag is 1 during encoding or decoding (S5201).

[0533] When the CABAC initialization flag is 1 (Yes in S5201), the three-dimensional data encoding device or three-dimensional data decoding device initializes the CABAC encoding / decoding device to a default state (S5202) and continues encoding or decoding.

[0534] The three-dimensional data encoding device or the three-dimensional data decoding device continues encoding or decoding without initialization when the CABAC initialization flag is not 1 (No in S5201).

[0535] That is, when initializing CABAC, set CABAC_init_flag=1 and initialize or re-initialize the encoding or decoding portion of CABAC. Additionally, when initializing, set the initial value (default state) of the context used for CABAC processing.

[0536] The encoding process is described. FIG. 77 is a block diagram showing the configuration of a first encoding unit (5200) included in a three-dimensional data encoding device according to the present embodiment. FIG. 78 is a block diagram showing the configuration of a division unit (5201) according to the present embodiment. FIG. 79 is a block diagram showing the configuration of a position information encoding unit (5202) and an attribute information encoding unit (5203) according to the present embodiment.

[0537] The first encoding unit (5200) generates encoded data (encoded stream) by encoding point cloud data using a first encoding method (GPCC ((Geometry based PCC)). This first encoding unit (5200) includes a splitting unit (5201), a plurality of position information encoding units (5202), a plurality of attribute information encoding units (5203), an additional information encoding unit (5204), and a multiplexing unit (5205).

[0538] The dividing unit (5201) generates multiple divided data by dividing the point cloud data. Specifically, the dividing unit (5201) generates multiple divided data by dividing the space of the point cloud data into multiple sub-spaces. Here, a sub-space is either a tile or a slice, or a combination of a tile and a slice. More specifically, the point cloud data includes location information, attribute information, and additional information. The dividing unit (5201) divides the location information into multiple divided location information and divides the attribute information into multiple divided attribute information. In addition, the dividing unit (5201) generates additional information regarding the division.

[0539] The dividing unit (5201) includes a tile dividing unit (5211) and a slice dividing unit (5212), as shown in FIG. 78. For example, the tile dividing unit (5211) divides a point group into tiles. The tile dividing unit (5211) may determine the quantization value used for each divided tile as tile additional information.

[0540] The slice division unit (5212) further divides the tile obtained by the tile division unit (5211) into slices. The slice division unit (5212) may determine the quantization value used for each divided slice as slice additional information.

[0541] A plurality of location information encoding units (5202) generate a plurality of encoded location information by encoding a plurality of divided location information. For example, the plurality of location information encoding units (5202) process the plurality of divided location information in parallel.

[0542] The position information encoding unit (5202) includes a CABAC initialization unit (5221) and an entropy encoding unit (5222), as shown in FIG. 79. The CABAC initialization unit (5221) initializes or re-initializes the CABAC according to the CABAC initialization flag. The entropy encoding unit (5222) encodes the divided position information by the CABAC.

[0543] A plurality of attribute information encoding units (5203) generates a plurality of encoded attribute information by encoding a plurality of divided attribute information. For example, the plurality of attribute information encoding units (5203) process a plurality of divided attribute information in parallel.

[0544] The attribute information encoding unit (5203) includes a CABAC initialization unit (5231) and an entropy encoding unit (5232), as shown in FIG. 79. The CABAC initialization unit (5221) initializes or re-initializes the CABAC according to the CABAC initialization flag. The entropy encoding unit (5232) encodes the partitioned attribute information by the CABAC.

[0545] The additional information encoding unit (5204) generates encoded additional information by encoding additional information included in the point cloud data and additional information regarding data division generated at the division unit (5201).

[0546] The multiplexer (5205) generates encoded data (encoded stream) by multiplexing a plurality of encoding location information, a plurality of encoding attribute information, and encoding additional information, and transmits the generated encoded data. In addition, the encoding additional information is used during decoding.

[0547] Additionally, FIG. 77 illustrates an example in which the number of location information encoding units (5202) and attribute information encoding units (5203) are each two, but the number of location information encoding units (5202) and attribute information encoding units (5203) may each be one or three or more. Also, multiple divided data may be processed in parallel within the same chip, such as multiple cores in a CPU, processed in parallel by cores of multiple chips, or processed in parallel by multiple cores of multiple chips.

[0548] Next, decoding processing will be described. FIG. 80 is a block diagram showing the configuration of the first decoding unit (5240). FIG. 81 is a block diagram showing the configuration of the location information decoding unit (5242) and the attribute information decoding unit (5243).

[0549] The first decoding unit (5240) restores point cloud data by decoding the encoded data (encoded stream) generated by encoding the point cloud data with the first encoding method (GPCC). This first decoding unit (5240) includes a demultiplexer (5241), a plurality of position information decoding units (5242), a plurality of attribute information decoding units (5243), an additional information decoding unit (5244), and a combination unit (5245).

[0550] The demultiplexer (5241) generates multiple encoding location information, multiple encoding attribute information, and encoding additional information by demultiplexing the encoding data (encoding stream).

[0551] A plurality of position information decoding units (5242) generate a plurality of quantized position information by decoding a plurality of encoded position information. For example, the plurality of position information decoding units (5242) process a plurality of encoded position information in parallel.

[0552] The position information decoding unit (5242) includes a CABAC initialization unit (5251) and an entropy decoding unit (5252), as shown in FIG. 81. The CABAC initialization unit (5251) initializes or re-initializes the CABAC according to the CABAC initialization flag. The entropy decoding unit (5252) decodes the position information using the CABAC.

[0553] The multiple attribute information decoding unit (5243) generates multiple divided attribute information by decoding multiple encoded attribute information. For example, the multiple attribute information decoding unit (5243) processes multiple encoded attribute information in parallel.

[0554] The attribute information decoding unit (5243) includes a CABAC initialization unit (5261) and an entropy decoding unit (5262), as shown in FIG. 81. The CABAC initialization unit (5261) initializes or re-initializes the CABAC according to the CABAC initialization flag. The entropy decoding unit (5262) ​​decodes the attribute information using the CABAC.

[0555] A plurality of additional information decoding units (5244) generate additional information by decoding the encoded additional information.

[0556] The combining unit (5245) generates location information by combining a plurality of segmented location information using additional information. The combining unit (5245) generates attribute information by combining a plurality of segmented attribute information using additional information. For example, the combining unit (5245) first generates point cloud data corresponding to a tile by combining decoded point cloud data for a slice using slice additional information. Next, the combining unit (5245) restores the original point cloud data by combining point cloud data corresponding to a tile using tile additional information.

[0557] Additionally, FIG. 80 illustrates an example in which the number of location information decoding units (5242) and attribute information decoding units (5243) are each two, but the number of location information decoding units (5242) and attribute information decoding units (5243) may each be one or three or more. Also, multiple divided data may be processed in parallel within the same chip, such as multiple cores within a CPU, processed in parallel by cores of multiple chips, or processed in parallel by multiple cores of multiple chips.

[0558] FIG. 82 is a flowchart showing an example of processing related to the initialization of CABAC in the encoding of position information or attribute information.

[0559] First, the three-dimensional data encoding device determines whether to initialize CABAC in the encoding of position information of the slice for each slice based on a predetermined condition (S5201).

[0560] When the three-dimensional data encoding device determines that it initializes CABAC (Yes in S5202), it determines the context initial value used for encoding position information (S5203). The context initial value is set to an initial value that takes into account the encoding characteristics. The initial value may be a predetermined value, or it may be adaptively determined according to the characteristics of the data within the slice.

[0561] Next, the three-dimensional data encoding device sets the CABAC initialization flag of the position information to 1 and sets the context initial value (S5204). When initializing CABAC, the initialization process is executed using the context initial value during the encoding of the position information.

[0562] Meanwhile, if the three-dimensional data encoding device determines that it does not initialize CABAC (No in S5202), it sets the CABAC initialization flag of the position information to 0 (S5205).

[0563] Next, the three-dimensional data encoding device determines whether to initialize CABAC in the encoding of attribute information of the slice for each slice based on a predetermined condition (S5206).

[0564] When the three-dimensional data encoding device determines that it initializes CABAC (Yes in S5207), it determines the context initial value used for encoding attribute information (S5208). The context initial value is set to an initial value that takes into account encoding characteristics. The initial value may be a predetermined value, or it may be adaptively determined according to the characteristics of the data within the slice.

[0565] Next, the three-dimensional data encoding device sets the CABAC initialization flag of the attribute information to 1 and sets the context initial value (S5209). When CABAC initialization is performed, the initialization process is executed using the context initial value during the encoding of the attribute information.

[0566] Meanwhile, if the three-dimensional data encoding device determines that it does not initialize CABAC (No in S5207), it sets the CABAC initialization flag of the attribute information to 0 (S5210).

[0567] In addition, in the flowchart of FIG. 82, the processing order of the processing related to position information and the processing related to attribute information may be reversed or parallel.

[0568] In addition, the flowchart in FIG. 82 shows processing in slice units as an example, but processing in tile units or other data units can also be processed in the same way as in slice units. That is, the slices in the flowchart of FIG. 82 can be read by changing them into tiles or other data units.

[0569] In addition, the specified conditions may be the same or different conditions regarding location information and attribute information.

[0570] FIG. 83 is a diagram showing an example of CABAC initialization timing in point cloud data as a bit stream.

[0571] Point cloud data includes location information and zero or more attribute information. That is, point cloud data may have no attribute information, or it may have multiple attribute information.

[0572] For example, for one three-dimensional point, as attribute information, there are cases where color information is present, cases where color information and reflection information are present, and cases where one or more color information is present, each associated with one or more viewpoint information.

[0573] In any configuration, the method described in this embodiment can be applied.

[0574] Next, the determination conditions for the initialization of CABAC will be explained.

[0575] If the following conditions are satisfied, the CABAC in the encoding of location information or attribute information may be initialized.

[0576] For example, CABAC may be initialized with the leading data of location information or attribute information (each attribute information if there are multiple). For example, CABAC may be initialized with the leading data of a PCC frame that can be decoded independently. That is, as shown in FIG. 83 (a), if a PCC frame can be decoded on a frame-by-frame basis, CABAC may be initialized with the leading data of the PCC frame.

[0577] Also, for example, as shown in Fig. 83 (b), in cases where inter-prediction is used between PCC frames and the frame cannot be decoded alone, CABAC may be initialized with the leading data of a random access unit (e.g., GOF).

[0578] Also, for example, as shown in Fig. 83 (c), CABAC may be initialized at the beginning of slice data divided into one or more parts, at the beginning of tile data divided into one or more parts, or at the beginning of other divided data.

[0579] Figure 83 (c) shows a tile as an example, and the same applies to a slice. At the beginning of the tile or slice, it may or may not be initialized.

[0580] FIG. 84 is a diagram showing the configuration of encoded data and the method of storing encoded data in a NAL unit.

[0581] Initialization information may be stored in the header of the encoded data or in the metadata. Additionally, initialization information may be stored in both the header and the metadata. The initialization information is, for example, caba_init_flag, the CABAC initial value, or an index of a table that can specify the initial value.

[0582] In this embodiment, the part described as being stored in metadata may be read as being stored in the header of the encoded data, and vice versa.

[0583] When initialization information is stored in the header of the encoded data, for example, it may be stored in the first NAL unit of the encoded data. Initialization information for the encoding of the position information is stored in the position information, and initialization information for the encoding of the attribute information is stored in the attribute information.

[0584] The cabac_init_flag for attribute information encoding and the cabac_init_flag for position information encoding may have the same value or different values. If they have the same value, the cabac_init_flag for position information and the cabac_init_flag for attribute information may be common. Furthermore, if they have different values, the cabac_init_flag for position information and the cabac_init_flag for attribute information shall each represent different values.

[0585] Initialization information may be stored in common metadata for location information and attribute information, stored in at least one of individual metadata for location information and individual metadata for attribute information, or stored in both common metadata and individual metadata. Additionally, a flag indicating whether it is listed in any one of the individual metadata for location information, individual metadata for attribute information, and common metadata may be used.

[0586] FIG. 85 is a flowchart showing an example of processing related to the initialization of CABAC in the decoding of location information or attribute information.

[0587] The three-dimensional data decoding device interprets the encoded data and obtains the CABAC initialization flag of the position information, the CABAC initialization flag of the attribute information, and the context initial value (S5211).

[0588] Next, the three-dimensional data decoder determines whether the CABAC initialization flag of the position information is 1 (S5212).

[0589] When the CABAC initialization flag of the position information is 1 (Yes in S5212), the three-dimensional data decoding device initializes the CABAC decoding of the position information encoding using the context initial value of the position information encoding (S5213).

[0590] Meanwhile, the three-dimensional data decoding device does not initialize CABAC decoding in position information encoding when the CABAC initialization flag of the position information is 0 (No in S5212) (S5214).

[0591] Next, the three-dimensional data decoding device determines whether the CABAC initialization flag of the attribute information is 1 (S5215).

[0592] When the CABAC initialization flag of the attribute information is 1 (Yes in S5215), the three-dimensional data decoding device initializes the CABAC decoding of the attribute information encoding using the context initial value of the attribute information encoding (S5216).

[0593] Meanwhile, the three-dimensional data decoding device does not initialize CABAC decoding in attribute information encoding when the CABAC initialization flag of the attribute information is 0 (No in S5215) (S5217).

[0594] In addition, in the flowchart of FIG. 85, the processing order of the processing related to position information and the processing related to attribute information may be reversed or parallel.

[0595] In addition, the flowchart in FIG. 85 is applicable to both slice division and tile division cases.

[0596] Next, the flow of encoding and decoding processing of point cloud data according to the present embodiment will be described. FIG. 86 is a flowchart of encoding processing of point cloud data according to the present embodiment.

[0597] First, the three-dimensional data encoding device determines the division method to be used (S5221). This division method includes whether to perform tile division or slice division. In addition, the division method may include the number of divisions when performing tile division or slice division, and the type of division. The type of division is a method based on object shape as described above, a method based on map information or location information, or a method based on data amount or throughput. In addition, the division method may be predetermined.

[0598] When tile division is performed (Yes in S5222), the three-dimensional data encoding device generates multiple tile location information and multiple tile attribute information by dividing location information and attribute information into tile units (S5223). Additionally, the three-dimensional data encoding device generates tile additional information related to tile division.

[0599] When slice division is performed (Yes in S5224), the three-dimensional data encoding device generates multiple divided location information and multiple divided attribute information by dividing multiple tile location information and multiple tile attribute information (or location information and attribute information) (S5225). Additionally, the three-dimensional data encoding device generates location slice additional information and attribute slice additional information related to the slice division.

[0600] Next, the three-dimensional data encoding device generates multiple encoded location information and multiple encoded attribute information by encoding each of the multiple segmented location information and multiple segmented attribute information (S5226). In addition, the three-dimensional data encoding device generates dependency relationship information.

[0601] Next, the three-dimensional data encoding device generates encoded data (encoded stream) by unitizing (multiplexing) a plurality of encoding position information, a plurality of encoding attribute information and additional information into NAL units (S5227). In addition, the three-dimensional data encoding device transmits the generated encoded data.

[0602] FIG. 87 is a flowchart illustrating an example of processing to determine the value of a CABAC initialization flag and update additional information in the division of a tile (S5222) or the division of a slice (S5225).

[0603] In steps S5222 and S5225, the location information and attribute information of the tiles and / or slices may be divided individually and independently in each method, or divided collectively in batches. As a result, additional information divided for each tile and / or slice is generated.

[0604] At this time, the three-dimensional data encoding device determines whether to set the CABAC initialization flag to 1 or 0 (S5231).

[0605] Then, the three-dimensional data encoding device updates additional information to include the determined CABAC initialization flag (S5232).

[0606] FIG. 88 is a flowchart showing an example of a CABAC initialization process in the processing of encoding (S5226).

[0607] The three-dimensional data encoding device determines whether the CABAC initialization flag is 1 (S5241).

[0608] The three-dimensional data encoding device re-initializes the CABAC encoding unit to the default state when the CABAC initialization flag is 1 (Yes in S5241) (S5242).

[0609] And, the three-dimensional data encoding device continues the encoding process until the stopping condition of the encoding process is satisfied, for example, until the data to be encoded is lost (S5243).

[0610] FIG. 89 is a flowchart of the decoding process of point cloud data according to the present embodiment. First, the three-dimensional data decoding device determines a division method by interpreting additional information related to the division method (tile additional information, position slice additional information and attribute slice additional information) included in the encoded data (encoded stream) (S5251). This division method includes whether to perform tile division or slice division. In addition, the division method may include the number of divisions when performing tile division or slice division, and the type of division, etc.

[0611] Next, the three-dimensional data decoding device generates segmentation position information and segmentation attribute information by decoding a plurality of encoding position information and a plurality of encoding attribute information included in the encoded data using dependency relationship information included in the encoded data (S5252).

[0612] When it is indicated that slice division is performed based on additional information (Yes in S5253), the three-dimensional data decoding device generates multiple tile location information and multiple tile attribute information by combining multiple division location information and multiple division attribute information based on location slice additional information and attribute slice additional information (S5254).

[0613] When it is indicated that tile division is performed based on additional information (Yes in S5255), the three-dimensional data decoding device generates position information and attribute information by combining a plurality of tile position information and a plurality of tile attribute information (a plurality of division position information and a plurality of division attribute information) based on the tile additional information (S5256).

[0614] FIG. 90 is a flowchart illustrating an example of a process for initializing a CABAC decoding unit in the combination of information divided per slice (S5254) or information divided per tile (S5256).

[0615] The position information and attribute information of slices or tiles may be combined using their respective methods, or combined using the same method.

[0616] The three-dimensional data decoding device decodes the CABAC initialization flag from the additional information of the encoding stream (S5261).

[0617] Next, the three-dimensional data decoder determines whether the CABAC initialization flag is 1 (S5262).

[0618] The three-dimensional data decoding device re-initializes the CABAC decoding unit to the default state when the CABAC initialization flag is 1 (Yes in S5262) ​​(S5263).

[0619] Meanwhile, if the CABAC initialization flag is not 1 (No in S5262), the three-dimensional data decoding device proceeds to step S5264 without re-initializing the CABAC decoding unit.

[0620] And, the three-dimensional data decoding device continues the decoding process until the condition for stopping the decoding process is satisfied, for example, until the data to be decoded is lost (S5264).

[0621] Next, other determination conditions for CABAC initialization will be explained.

[0622] Whether to initialize the encoding of location information or attribute information may be determined by considering the encoding efficiency of data units such as tiles or slices. In this case, CABAC may be initialized for the leading data of a tile or slice that satisfies a predetermined condition.

[0623] Next, the determination conditions for CABAC initialization in the encoding of position information will be explained.

[0624] The three-dimensional data encoding device may, for example, determine the density of point cloud data for each slice, that is, the number of points per unit area belonging to the slice, and compare the data density of another slice with the data density of the slice in question; if the change in data density does not exceed a predetermined condition, it may determine that it is more efficient to not initialize CABAC and decide not to initialize CABAC. On the other hand, if the change in data density does not satisfy the predetermined condition, the three-dimensional data encoding device may determine that it is more efficient to initialize CABAC and decide to initialize CABAC.

[0625] Here, another slice may be, for example, a slice one position ahead in the order of decoding processing, or a spatially adjacent slice. In addition, the three-dimensional data encoding device may determine whether to initialize CABAC based on whether the data density of the slice is a predetermined data density, without comparing it with the data density of another slice.

[0626] When the three-dimensional data encoding device determines that it initializes CABAC, it determines the context initial value to be used for encoding position information. The context initial value is set to an initial value that provides good encoding characteristics based on data density. The three-dimensional data encoding device may maintain a table of initial values ​​for data density in advance and select the optimal initial value from the table.

[0627] In addition, the three-dimensional data encoding device may determine whether to initialize CABAC based on the number of points, point distribution, point skewness, etc., not limited to examples such as slice density. Alternatively, the three-dimensional data encoding device may determine whether to initialize CABAC based on feature quantities or the number of feature points obtained from point information, or the recognized object. In that case, the determination criteria may be maintained in memory in advance as a table corresponding to feature quantities or the number of feature points obtained from point information, or the object recognized based on point information.

[0628] The three-dimensional data encoding device may, for example, determine an object in the location information of map information and determine whether to initialize CABAC based on the object based on the location information, or determine whether to initialize CABAC based on information or feature quantity obtained by projecting the three-dimensional data onto two dimensions.

[0629] Next, the determination conditions for CABAC initialization in the encoding of attribute information will be explained.

[0630] A three-dimensional data encoding device may, for example, compare the color characteristics of a preceding slice with the color characteristics of the slice in question, and if the change in color characteristics satisfies a predetermined condition, determine that it is more efficient to not initialize CABAC and decide not to initialize CABAC. On the other hand, if the change in color characteristics does not satisfy a predetermined condition, the three-dimensional data encoding device may determine that it is more efficient to initialize CABAC and decide to initialize it. Color characteristics include, for example, luminance, chromaticity, saturation, histograms thereof, color continuity, etc.

[0631] Here, another slice may be, for example, a slice one position ahead in the order of decoding processing, or a spatially adjacent slice. In addition, the three-dimensional data encoding device may determine whether to initialize CABAC based on whether the data density of the slice is a predetermined data density, without comparing it with the data density of another slice.

[0632] When the three-dimensional data encoding device determines that it initializes CABAC, it determines the context initial value to be used for encoding attribute information. The context initial value is set to an initial value that provides good encoding characteristics based on data density. The three-dimensional data encoding device may maintain a table of initial values ​​for data density in advance and select the optimal initial value from the table.

[0633] If the attribute information is reflectance, the three-dimensional data encoding device may determine whether to initialize CABAC based on the information based on the reflectance.

[0634] When a three-dimensional data encoding device has multiple attribute information in a three-dimensional point, it may determine initialization information based on each attribute information independently for each attribute information, or determine initialization information for multiple attribute information based on one of the attribute information, or determine initialization information for multiple attribute information using multiple attribute information.

[0635] An example has been explained in which the initialization information for location information is determined based on location information, and the initialization information for attribute information is determined based on attribute information. The initialization information for location information and attribute information may be determined based on location information, the initialization information for location information and attribute information may be determined based on attribute information, or the initialization information for location information and attribute information may be determined based on both types of information.

[0636] The three-dimensional data encoding device may determine initialization information based on the result of simulating encoding efficiency in advance by, for example, setting cabac_init_flag to on or off, or using one or more initial values ​​from an initial value table.

[0637] When a three-dimensional data encoding device determines a data division method into slices or tiles based on position information or attribute information, it may determine initialization information based on information such as that based on the determination of the division method.

[0638] Fig. 91 is a drawing showing examples of tiles and slices.

[0639] For example, a slice in a single tile containing a portion of PCC data is identified as shown in the legend. The CABAC initialization flag can be used to determine whether context re-initialization is required for consecutive slices. For example, in FIG. 91, if slice data divided by object (moving object, sidewalk, building, tree, other object) is included in a single tile, the CABAC initialization flag for slices of moving object, sidewalk, and tree is set to 1, and the CABAC initialization flag for slices of building or other objects is set to 0. This means that, for example, if both the sidewalk and the building are dense permanent structures and likely have the same coding efficiency, coding efficiency may be improved by not re-initializing the CABAC between the slices of the sidewalk and the building. On the other hand, if the density and coding efficiency may differ significantly between the building and the tree, coding efficiency may be improved by initializing the CABAC between the slices of the building and the tree.

[0640] FIG. 92 is a flowchart illustrating an example of a method for initializing CABAC and determining the context initial value.

[0641] First, the three-dimensional data encoding device divides the point cloud data into slices based on the object determined from the position information (S5271).

[0642] Next, the three-dimensional data encoding device determines whether to initialize CABAC for position information encoding and attribute information encoding for each slice based on the data density of the object in the slice (S5272). That is, the three-dimensional data encoding device determines CABAC initialization information (CABAC initialization flag) for position information encoding and attribute information encoding based on position information. The three-dimensional data encoding device determines the initialization with good encoding efficiency, for example, based on point cloud data density. In addition, the CABAC initialization information may appear in a common cabac_init_flag for position information and attribute information.

[0643] Next, if the three-dimensional data encoding device determines that it initializes CABAC (Yes in S5273), it determines the context initial value for encoding position information (S5274).

[0644] Next, the three-dimensional data encoding device determines the context initial value for encoding attribute information (S5275).

[0645] Next, the three-dimensional data encoding device sets the CABAC initialization flag of the position information to 1 and sets the context initial value of the position information, and also sets the CABAC initialization flag of the attribute information to 1 and sets the context initial value of the attribute information (S5276). In addition, when initializing CABAC, the three-dimensional data encoding device performs initialization processing using the context initial value in each of the encoding of the position information and the encoding of the attribute information.

[0646] Meanwhile, if the three-dimensional data encoding device determines that it does not initialize CABAC (No in S5273), it sets the CABAC initialization flag of the position information to 0 and sets the CABAC initialization flag of the attribute information to 0 (S5277).

[0647] FIG. 93 is a diagram showing an example of a map viewed from the top plane being divided into tiles using point cloud data obtained from LiDAR. FIG. 94 is a flowchart showing another example of a method for initializing CABAC and determining context initial values.

[0648] A three-dimensional data encoding device divides point cloud data into one or more tiles based on location information in large-scale map data, using a two-dimensional division method when viewed from the top surface (S5281). The three-dimensional data encoding device may divide the point cloud data into square regions, for example, as shown in FIG. 93. In addition, the three-dimensional data encoding device may divide the point cloud data into tiles of various shapes and sizes. The division of tiles may be performed using one or more predetermined methods or may be performed adaptively.

[0649] Next, the three-dimensional data encoding device determines an object within each tile and determines whether to initialize CABAC during the encoding of position information or attribute information of the said tile (S5282). Additionally, in the case of slice division, the three-dimensional data encoding device recognizes an object (tree, person, moving object, building) and determines the slice division and initial value according to the object.

[0650] When the three-dimensional data encoding device determines that it initializes CABAC (Yes in S5283), it determines the context initial value for encoding position information (S5284).

[0651] Next, the three-dimensional data encoding device determines the context initial value for encoding attribute information (S5285).

[0652] In steps S5284 and S5285, as an initial value, the initial value of a tile having a specific encoding characteristic may be stored and used as the initial value of a tile having the same encoding characteristic.

[0653] Next, the three-dimensional data encoding device sets the CABAC initialization flag of the position information to 1 and sets the context initial value of the position information, and also sets the CABAC initialization flag of the attribute information to 1 and sets the context initial value of the attribute information (S5286). In addition, when initializing CABAC, the three-dimensional data encoding device performs initialization processing using the context initial value in each of the encoding of the position information and the encoding of the attribute information.

[0654] Meanwhile, if the three-dimensional data encoding device determines that it does not initialize CABAC (No in S5283), it sets the CABAC initialization flag of the position information to 0 and sets the CABAC initialization flag of the attribute information to 0 (S5287).

[0655] (Embodiment 8)

[0656] The quantization parameters are explained below.

[0657] Slices and tiles are used to segment point cloud data based on its characteristics and location. Here, depending on hardware limitations and real-time processing requirements, the quality required for each segmented point cloud data may differ. For example, when segmenting and encoding data by object, slice data containing plants may have its resolution (quality) reduced by quantization because it is less important. On the other hand, important slice data can be maintained at high resolution (quality) by setting the quantization value to a low value. Quantization parameters are used to enable such control of quantization values.

[0658] Here, the data to be quantized, the scale used for quantization, and the quantized data, which is the result produced by quantization, are represented by the following (Equation G1) and (Equation G2).

[0659] Quantized data = data / scale (Equation G1)

[0660] Data = Quantized Data * Scale (Equation G2)

[0661] FIG. 95 is a diagram for explaining the processing of a quantization unit (5323) that quantizes data and an inverse quantization unit (5333) that inversely quantizes the quantized data.

[0662] The quantization unit (5323) quantizes the data using a scale, that is, by performing a process using Equation G1, to produce quantized data in which the data is quantized.

[0663] The inverse quantization unit (5333) inversely quantizes the quantized data using a scale, that is, by performing a process using Equation G2, it produces the quantized data in which the quantized data has been inversely quantized.

[0664] Also, the scale and the quantization value (QP(Quantization Parameter) value) are expressed by the following (Equation G3).

[0665] Quantization value (QP value) = log(scale) (Equation G3)

[0666] Quantization value (QP value) = Default value (reference value) + Quantization delta (difference information) (Equation G4)

[0667] Also, these parameters are collectively called quantization parameters.

[0668] For example, as shown in FIG. 96, the quantization value is a value based on the default value and is calculated by adding the quantization delta to the default value. If the quantization value is smaller than the default value, the quantization delta becomes a negative value. If the quantization value is larger than the default value, the quantization delta becomes a positive value. If the quantization value is equal to the default value, the quantization delta becomes 0. If the quantization delta is 0, the quantization delta is not required.

[0669] The encoding process is described. FIG. 97 is a block diagram showing the configuration of a first encoding unit (5300) included in a three-dimensional data encoding device according to the present embodiment. FIG. 98 is a block diagram showing the configuration of a division unit (5301) according to the present embodiment. FIG. 99 is a block diagram showing the configuration of a position information encoding unit (5302) and an attribute information encoding unit (5303) according to the present embodiment.

[0670] The first encoding unit (5300) generates encoded data (encoded stream) by encoding point cloud data using a first encoding method (GPCC (Geometry based PCC)). This first encoding unit (5300) includes a splitting unit (5301), a plurality of position information encoding units (5302), a plurality of attribute information encoding units (5303), an additional information encoding unit (5304), and a multiplexing unit (5305).

[0671] The dividing unit (5301) generates multiple divided data by dividing the point cloud data. Specifically, the dividing unit (5301) generates multiple divided data by dividing the space of the point cloud data into multiple sub-spaces. Here, a sub-space is either a tile or a slice, or a combination of a tile and a slice. More specifically, the point cloud data includes location information, attribute information, and additional information. The dividing unit (5301) divides the location information into multiple divided location information and divides the attribute information into multiple divided attribute information. In addition, the dividing unit (5301) generates additional information related to the division.

[0672] The dividing unit (5301) includes a tile dividing unit (5311) and a slice dividing unit (5312), as shown in FIG. 98. For example, the tile dividing unit (5311) divides a point group into tiles. The tile dividing unit (5311) may determine the quantization value used for each divided tile as tile additional information.

[0673] The slice division unit (5312) further divides the tile obtained by the tile division unit (5311) into slices. The slice division unit (5312) may determine the quantization value used for each divided slice as slice additional information.

[0674] A plurality of location information encoding units (5302) generate a plurality of encoded location information by encoding a plurality of divided location information. For example, the plurality of location information encoding units (5302) process the plurality of divided location information in parallel.

[0675] The position information encoding unit (5302) includes a quantization value calculation unit (5321) and an entropy encoding unit (5322), as shown in FIG. 99. The quantization value calculation unit (5321) obtains a quantization value (quantization parameter) of the segmented position information to be encoded. The entropy encoding unit (5322) calculates quantized position information by quantizing the segmented position information using the quantization value (quantization parameter) obtained by the quantization value calculation unit (5321).

[0676] A plurality of attribute information encoding units (5303) generates a plurality of encoded attribute information by encoding a plurality of divided attribute information. For example, the plurality of attribute information encoding units (5303) process a plurality of divided attribute information in parallel.

[0677] The attribute information encoding unit (5303) includes a quantization value calculation unit (5331) and an entropy encoding unit (5332), as shown in FIG. 99. The quantization value calculation unit (5331) obtains a quantization value (quantization parameter) of the divided attribute information to be encoded. The entropy encoding unit (5332) calculates quantized attribute information by quantizing the divided attribute information using the quantization value (quantization parameter) obtained by the quantization value calculation unit (5331).

[0678] The additional information encoding unit (5304) generates encoded additional information by encoding additional information included in the point cloud data and additional information related to data division generated at the division unit (5301).

[0679] The multiplexer (5305) generates encoded data (encoded stream) by multiplexing a plurality of encoding location information, a plurality of encoding attribute information, and encoding additional information, and transmits the generated encoded data. In addition, the encoding additional information is used during decoding.

[0680] Additionally, FIG. 97 illustrates an example in which the number of location information encoding units (5302) and attribute information encoding units (5303) are each two, but the number of location information encoding units (5302) and attribute information encoding units (5303) may each be one or three or more. Also, multiple divided data may be processed in parallel within the same chip, such as multiple cores within a CPU, processed in parallel by cores of multiple chips, or processed in parallel by multiple cores of multiple chips.

[0681] Next, decoding processing will be described. FIG. 100 is a block diagram showing the configuration of the first decoding unit (5340). FIG. 101 is a block diagram showing the configuration of the location information decoding unit (5342) and the attribute information decoding unit (5343).

[0682] The first decoding unit (5340) restores point cloud data by decoding the encoded data (encoded stream) generated by encoding the point cloud data with the first encoding method (GPCC). This first decoding unit (5340) includes a demultiplexer (5341), a plurality of position information decoding units (5342), a plurality of attribute information decoding units (5343), an additional information decoding unit (5344), and a combination unit (5345).

[0683] The demultiplexer (5341) generates multiple encoding location information, multiple encoding attribute information, and encoding additional information by demultiplexing the encoding data (encoding stream).

[0684] A plurality of position information decoding units (5342) generate a plurality of quantized position information by decoding a plurality of encoded position information. For example, the plurality of position information decoding units (5342) process a plurality of encoded position information in parallel.

[0685] The position information decoding unit (5342) includes a quantization value calculation unit (5351) and an entropy decoding unit (5352), as shown in FIG. 101. The quantization value calculation unit (5351) obtains a quantization value of the quantized position information. The entropy decoding unit (5352) calculates position information by inversely quantizing the quantized position information using the quantization value obtained by the quantization value calculation unit (5351).

[0686] The multiple attribute information decoding unit (5343) generates multiple divided attribute information by decoding multiple encoded attribute information. For example, the multiple attribute information decoding unit (5343) processes multiple encoded attribute information in parallel.

[0687] The attribute information decoding unit (5343) includes a quantization value calculation unit (5361) and an entropy decoding unit (5362), as shown in FIG. 101. The quantization value calculation unit (5361) obtains a quantization value of the quantized attribute information. The entropy decoding unit (5362) calculates the attribute information by inversely quantizing the quantized attribute information using the quantization value obtained by the quantization value calculation unit (5361).

[0688] A plurality of additional information decoding units (5344) generate additional information by decoding the encoded additional information.

[0689] The combining unit (5345) generates location information by combining a plurality of segmented location information using additional information. The combining unit (5345) generates attribute information by combining a plurality of segmented attribute information using additional information. For example, the combining unit (5345) first generates point cloud data corresponding to a tile by combining decoded point cloud data for a slice using slice additional information. Next, the combining unit (5345) restores the original point cloud data by combining point cloud data corresponding to a tile using tile additional information.

[0690] Additionally, FIG. 100 illustrates an example in which the number of location information decoding units (5342) and attribute information decoding units (5343) are each 2, but the number of location information decoding units (5342) and attribute information decoding units (5343) may each be 1 or 3 or more. Also, multiple divided data may be processed in parallel within the same chip, such as multiple cores in a CPU, processed in parallel by cores of multiple chips, or processed in parallel by multiple cores of multiple chips.

[0691] [Method for Determining Quantization Parameters]

[0692] FIG. 102 is a flowchart showing an example of processing regarding the determination of a quantization parameter value (QP value) in the encoding of position information (geometry) or attribute information (attribute).

[0693] The QP value is determined, for example, by considering encoding efficiency for each data unit of location information or each data unit of attribute information constituting the PCC frame. If the data unit is a divided tile unit or a divided slice unit, the QP value is determined for the divided data unit by considering the encoding efficiency of the divided data unit. Alternatively, the QP value may be determined for the data unit before division.

[0694] As shown in FIG. 102, the three-dimensional data encoding device determines a QP value to be used for encoding position information (S5301). The three-dimensional data encoding device may determine the QP value for each of the divided multiple slices according to a predetermined method. Specifically, the three-dimensional data encoding device determines the QP value based on the characteristics or quality of the position information data. For example, the three-dimensional data encoding device may determine the density of the point cloud data for each data unit, that is, the number of points per unit area belonging to the slice, and determine a value corresponding to the density of the point cloud data as the QP value. Alternatively, the three-dimensional data encoding device may determine a value corresponding to the QP value based on the number of points in the point cloud data, the distribution of points, the bias of points, or the amount of features obtained from the point information, the number of feature points, or the recognized object. In addition, the three-dimensional data encoding device may determine an object in the position information of a map and determine a QP value based on the object based on the position information, or determine a QP value based on information obtained by projecting a three-dimensional point cloud onto two dimensions or a feature quantity. The corresponding QP value may be maintained in memory in advance as a table corresponding to the density, number of points, distribution of points, or bias of points in the point cloud data. In addition, the corresponding QP value may be maintained in memory in advance as a table corresponding to a feature quantity or number of feature points obtained from the point information, or an object recognized based on the point information. In addition, the corresponding QP value may be determined based on the result of simulating the encoding rate, etc., with various QP values ​​when encoding the position information of the point cloud data.

[0695] Next, the three-dimensional data encoding device determines a reference value (default value) and difference information (quantization delta) of the QP value of the position information (S5302). Specifically, the three-dimensional data encoding device determines the reference value and difference information to be transmitted using the determined QP value and a predetermined method, and sets (adds) the determined reference value and difference information to at least one of the additional information and the data header.

[0696] Next, the three-dimensional data encoding device determines a QP value to be used for encoding attribute information (S5303). The three-dimensional data encoding device may determine the QP value for each of the divided multiple slices according to a predetermined method. Specifically, the three-dimensional data encoding device determines the QP value based on the characteristics or quality of the attribute information data. For example, the three-dimensional data encoding device may determine the QP value based on the characteristics of the attribute information for each data unit. Color characteristics include, for example, luminance, chromaticity, saturation, histograms thereof, color continuity, etc. If the attribute information is reflectance, it may be determined based on information based on reflectance. For example, when the three-dimensional data encoding device detects a face as an object from point cloud data, it may determine a QP value of good quality for the point cloud data constituting the object from which the face was detected. In this way, the three-dimensional data encoding device may determine the QP value for the point cloud data constituting the object according to the type of object.

[0697] In addition, when there are multiple attribute informations in a three-dimensional point, the three-dimensional data encoding device may determine a QP value based on each attribute information independently for each attribute information, or determine a QP value of multiple attribute information based on one of the attribute informations, or determine a QP value of multiple attribute information using multiple attribute information.

[0698] Next, the three-dimensional data encoding device determines a reference value (default value) and difference information (quantization delta) of the QP value of the attribute information (S5304). Specifically, the three-dimensional data encoding device determines the reference value and difference information to be transmitted using the determined QP value and a predetermined method, and sets (adds) the determined reference value and difference information to at least one of the additional information and the data header.

[0699] Then, the three-dimensional data encoding device quantizes and encodes the position information and attribute information based on the QP values ​​of the determined position information and attribute information, respectively (S5305).

[0700] In addition, examples have been described in which the QP value of location information is determined based on location information and the QP value of attribute information is determined based on attribute information, but this is not limited thereto. For example, the QP values ​​of location information and attribute information may be determined based on location information, may be determined based on attribute information, or may be determined based on location information and attribute information.

[0701] In addition, the QP values ​​of location information and attribute information may be adjusted by considering the balance between the quality of location information and the quality of attribute information in the point cloud data. For example, the QP values ​​of location information and attribute information may be determined such that the quality of location information is set high, and the quality of attribute information is set lower than the quality of location information. For example, the QP value of attribute information may be determined to satisfy the constrained condition that it is greater than or equal to the QP value of location information.

[0702] Additionally, the QP value may be adjusted so that the encoded data falls within a predetermined range of rates. For example, if the amount of code in the encoding of a preceding data unit is likely to exceed the predetermined rate, that is, if the difference to the predetermined rate is less than the first difference, the QP value may be adjusted so that the encoding quality of the data unit is degraded to less than the first difference. On the other hand, if the difference to the predetermined rate is greater than the second difference, which is greater than the first difference, and there is a sufficiently large difference, the QP value may be adjusted so that the encoding quality of the data unit is improved. Adjustment between data units may be, for example, between PCC frames, or between tiles or slices. The adjustment of the QP value of the attribute information may be adjusted based on the encoding rate of the position information.

[0703] In addition, in the flowchart of FIG. 102, the processing order of processing related to position information and processing related to attribute information may be reversed or parallel.

[0704] In addition, the flowchart in FIG. 102 shows processing in slice units as an example, but processing in tile units or other data units can also be processed in the same way as in slice units. That is, the slices in the flowchart of FIG. 102 can be read by changing them into tiles or other data units.

[0705] FIG. 103 is a flowchart showing an example of decoding processing of location information and attribute information.

[0706] As shown in FIG. 103, the three-dimensional data decoding device acquires reference value and difference information representing the QP value of position information and reference value and difference information representing the QP value of attribute information (S5311). Specifically, the three-dimensional data decoding device interprets either or both of the transmitted metadata and the header of the encoded data and acquires reference value and difference information for deriving the QP value.

[0707] Next, the three-dimensional data decoding device derives a QP value based on a predetermined method using the acquired reference value and difference information (S5312).

[0708] And, the three-dimensional data decoding device acquires quantized position information and decodes the position information by inversely quantizing the quantized position information using the derived QP value (S5313).

[0709] Next, the three-dimensional data decoding device acquires quantization attribute information and decodes attribute information by inversely quantizing the quantization attribute information using the derived QP value (S5314).

[0710] Next, the transmission method of quantization parameters will be explained.

[0711] FIG. 104 is a diagram for explaining a first example of a method for transmitting quantization parameters. FIG. 104 (a) is a diagram showing an example of a relationship of QP values.

[0712] In FIG. 104, Q G and Q A and represent, respectively, the absolute value of the QP value used for encoding position information, and the absolute value of the QP value used for encoding attribute information. Q G is an example of a first quantization parameter used to quantize the position information of each of a plurality of three-dimensional points. Also, Δ(Q A , Q G ) is, Q A Q used to derive G It represents difference information indicating the difference with. That is, Q A ne, Q G and Δ(Q A , Q G It is derived using ). In this way, the QP value is transmitted by dividing it into a reference value (absolute value) and difference information (relative value). Also, in decoding, the desired QP value is derived from the transmitted reference value and difference information.

[0713] For example, in FIG. 104 (a), the absolute value Q G and difference information Δ(Q A , Q G ) is transmitted, and in decoding, as indicated by the following (Equation G5), Q A ne, Q G Δ(Q A , Q G It is derived by adding ).

[0714] Q A =Q G +Δ(Q A , Q G ) (Equation G5)

[0715] Using FIGS. 104 (b) and (c), a method for transmitting QP values ​​when slicing point cloud data consisting of position information and attribute information is explained. FIGS. 104 (b) is a diagram showing a first example of the relationship between the reference value and difference information of each QP value. FIGS. 104 (c) is a diagram showing a first example of the transmission order of QP values, position information, and attribute information.

[0716] QP values ​​are broadly divided into frame-unit QP values ​​(frame QP) and data-unit QP values ​​(data QP) of the PCC for each location information and for each attribute information. The data-unit QP values ​​are QP values ​​used for encoding, determined in step S5301 of FIG. 102.

[0717] Here, Q is the QP value used for encoding position information at the PCC frame level. G Using as the reference value, the QP value in data units is Q G It generates and transmits difference information representing the difference from.

[0718] Q G : QP value of position information encoding in the PCC frame... transmitted as reference value "1." using GPS

[0719] Q A: QP value of attribute information encoding in a PCC frame···Q using APS G Distinct information "2." indicating the difference from

[0720] Q Gs1 , Q Gs2 : QP value of position information encoding in slice data… using the header of the position information encoding data, Q G Distinct information "3." and "5." representing the difference from

[0721] Q As1 , Q As2 : QP value of attribute information encoding in slice data… using the header of the attribute information encoding data, Q A Distinct information "4." and "6." representing the difference from

[0722] In addition, the information used to derive the frame QP is recorded in the frame-related metadata (GPS, APS), and the information used to derive the data QP is recorded in the data-related metadata (header of the encoded data).

[0723] In this way, the data QP is generated and transmitted as difference information representing the difference from the frame QP. Accordingly, the amount of data in the data QP can be reduced.

[0724] The first decoding unit (5340) obtains reference values ​​and difference information corresponding to each encoded data by referring to the metadata indicated by the arrow (c) of FIG. 104. Then, the first decoding unit (5340) derives QP values ​​corresponding to the encoded data to be decoded based on the obtained reference values ​​and difference information.

[0725] The first decoding unit (5340), for example, obtains reference information “1.” and difference information “2.” and “6.” indicated by arrows in (c) of FIG. 104 from metadata or a header, and adds difference information “2.” and “6.” to reference information “1.” as indicated by the following (Equation G6), thereby A s2 Derive the QP value of.

[0726] Q AS2 =Q G +Δ(Q A , Q G )+Δ(Q As2 , Q A ) (Equation G6)

[0727] Point cloud data includes location information and zero or more attribute information. That is, point cloud data may have no attribute information, or it may have multiple attribute information.

[0728] For example, for one three-dimensional point, as attribute information, there are cases where color information is present, cases where color information and reflection information are present, and cases where one or more color information is present, each associated with one or more viewpoint information.

[0729] Here, an example having two color information and reflection information is explained using FIG. 105. FIG. 105 is a diagram for explaining a second example of a method for transmitting quantization parameters. FIG. 105 (a) is a diagram showing a second example of the relationship between the reference value of each QP value and the difference information. FIG. 105 (b) is a diagram showing a second example of the transmission order of QP values, position information, and attribute information.

[0730] Q G is an example of a first quantization parameter, similar to Fig. 104.

[0731] Each of the two color information is represented by luminance (luminance) Y and color difference (chroma) Cb and Cr. Q is the QP value used for encoding the luminance Y1 of the first color. Y1is the reference value Q G and Δ(Q representing that difference Y1 , Q G It is derived using ). Luminance Y1 is an example of the first luminance, and Q Y1 Δ(Q) is an example of a second quantization parameter used to quantize luminance Y1 as the first luminance. Y1 , Q G ) is the difference information "2."

[0732] Also, Q, the QP value used for encoding the color difference Cb1, Cr1 of the first color. Cb1 , Q Cr1 ≠ QY1 and Δ(Q representing the difference thereof), respectively. Cb1 , Q Y1 ), Δ(Q Cr1 , Q Y1 It is derived using ). Color differences Cb1, Cr1 are examples of the first color difference, and Q Cb1 , Q Cr1 Δ(Q) is an example of a third quantization parameter used to quantize the color differences Cb1, Cr1 as the first color difference. Cb1 , Q Y1 ) is the difference information "3.", and Δ(Q Cr1 , Q Y1 ) is the difference information "4." Δ(Q Cb1 , Q Y1 ) and Δ(Q Cr1 , Q Y1 ) are, respectively, examples of the first degree.

[0733] Also, Q Cb1 and Q Cr1 For, the same value may be used, or a common value may be used. If a common value is used, Q Cb1 and Q Cr1 Since only one side needs to be used, the other side is not necessary.

[0734] Also, Q, which is the QP value used for encoding the luminance Y1D of the first color in the slice data. Y1D ne, Q Y1and Δ(Q representing the difference Y1D , Q Y1 It is derived using ). The luminance Y1D of the first color in the slice data is an example of the first luminance of one or more three-dimensional points included in the subspace, and Q Y1D Δ(Q) is an example of a fifth quantization parameter used to quantize luminance Y1D. Y1D , Q Y1 ) is the difference information “10.” and is an example of a second difference.

[0735] Likewise, Q, the QP value used for encoding the first color difference Cb1D and Cr1D in the slice data. Cb1D , Q Cr1D is, respectively, Q Cb1 , Q Cr1 and Δ(Q representing the difference Cb1D , Q Cb1 ), Δ(Q Cr1D , Q Cr1 It is derived using ). The first color difference Cb1D, Cr1D in the slice data is an example of the first color difference of one or more three-dimensional points included in the subspace, and Q Cb1D , Q Cr1D Δ(Q) is an example of the sixth quantization parameter used to quantize the color difference Cb1D, Cr1D. Cb1D , Q Cb1 ) is the difference information "11.", and Δ(Q Cr1D , Q Cr1 ) is the difference information 「12.」. Δ(Q Cb1D , Q Cb1 ) and Δ(Q Cr1D , Q Cr1 ) is an example of a third order.

[0736] Since the relationship of QP values ​​in the first color can be said to be the same for the second color, the explanation is omitted.

[0737] Q, the QP value used for encoding the reflectance R R is the reference value Q Gand Δ(Q representing that difference R , Q G It is derived using ). Q R Δ(Q) is an example of a fourth quantization parameter used to quantize the reflectance R. R , Q G ) is the difference information "8."

[0738] Also, Q, the QP value used for encoding the reflectance RD in the slice data. RD ne, Q R and Δ(Q representing the difference RD , Q R It is derived using Δ(Q RD , Q R ) is the difference information "16."

[0739] As such, difference information “9.” through “16.” represents the difference information between the data QP and the frame QP.

[0740] In addition, for example, if the values ​​of the data QP and the frame QP are the same, the difference information may be set to 0, or the difference information may be considered as 0 by not transmitting it.

[0741] The first decoding unit (5340), for example, when decoding the color difference Cr2 of the second color, obtains reference information “1.” and difference information “5.”, “7.” and “15.” indicated by the arrow in (b) of FIG. 105 from metadata or a header, and derives the QP value of the color difference Cr2 by adding the difference information “5.”, “7.” and “15.” to the reference information “1.” as indicated by the following (Equation G7).

[0742] Q Cr2D =Q G +Δ(Q Y2 , Q G )+Δ(Q Cr2 , Q Y2 )+Δ(Q Cr2D , Q Cr2 ) (Equation G7)

[0743] Next, using FIG. 106, an example of dividing location information and attribute information into two tiles and then into two slices will be explained. FIG. 106 is a diagram for explaining a third example of a method for transmitting quantization parameters. FIG. 106 (a) is a diagram showing a third example of the relationship between the reference value of each QP value and the difference information. FIG. 106 (b) is a diagram showing a third example of the transmission order of QP values, location information, and attribute information. FIG. 106 (c) is a diagram for explaining the intermediate generated value of the difference information in the third example.

[0744] When dividing into multiple tiles and then further dividing into multiple slices, as shown in (c) of FIG. 106, the QP value (Q) for each tile after dividing into tiles At1 ) and difference information Δ(Q At1 , Q A ) is generated as an intermediate generated value. And, after dividing into slices, the QP value (Q for each slice) At1s1 , Q At1s2 ) and difference information(Δ(Q At1s1 , Q At1 ), Δ(Q At1s2 , Q At1 )) is generated.

[0745] In this case, for example, the difference information “4.” in Fig. 106 (a) is derived by the following (Equation G8).

[0746] Δ(Q At1s1 , Q A )=Δ(Q At1 , Q A )+Δ(Q At1s1 , Q At1 ) (Equation G8)

[0747] The first decoding unit (5340) is, for example, attribute information A of slice 1 in tile 2. t2s1When decoding, reference information “1.” and difference information “2.” and “8.” indicated by the arrow in FIG. 106 (b) are obtained from metadata or a header, and by adding difference information “2.” and “8.” to reference information “1.” as shown in the following (Equation G9), attribute information A t2s1 Derive the QP value of.

[0748] Q At2s1 =Q G +Δ(Q At2s1 , Q A )+Δ(Q A , Q G ) (Equation G9)

[0749] Next, the flow of encoding and decoding processing of point cloud data according to the present embodiment will be described. FIG. 107 is a flowchart of encoding processing of point cloud data according to the present embodiment.

[0750] First, the three-dimensional data encoding device determines the division method to be used (S5321). This division method includes whether to perform tile division or slice division. In addition, the division method may include the number of divisions when performing tile division or slice division, and the type of division. The type of division is a method based on object shape as described above, a method based on map information or location information, or a method based on data amount or throughput. In addition, the division method may be predetermined.

[0751] When tile division is performed (Yes in S5322), the three-dimensional data encoding device generates multiple tile location information and multiple tile attribute information by dividing location information and attribute information into tile units (S5323). Additionally, the three-dimensional data encoding device generates tile additional information related to tile division.

[0752] When slice division is performed (Yes in S5324), the three-dimensional data encoding device generates multiple divided location information and multiple divided attribute information by dividing multiple tile location information and multiple tile attribute information (or location information and attribute information) (S5325). Additionally, the three-dimensional data encoding device generates location slice additional information and attribute slice additional information related to the slice division.

[0753] Next, the three-dimensional data encoding device generates multiple encoded location information and multiple encoded attribute information by encoding each of the multiple segmented location information and multiple segmented attribute information (S5326). In addition, the three-dimensional data encoding device generates dependency relationship information.

[0754] Next, the three-dimensional data encoding device generates encoded data (encoded stream) by unitizing (multiplexing) a plurality of encoding position information, a plurality of encoding attribute information and additional information into NAL units (S5327). In addition, the three-dimensional data encoding device transmits the generated encoded data.

[0755] FIG. 108 is a flowchart illustrating an example of a process for determining a QP value and updating additional information in the division of a tile (S5323) or the division of a slice (S5325).

[0756] In steps S5323 and S5325, the location information and attribute information of the tiles and / or slices may be divided individually and independently in each method, or divided collectively in batches. As a result, additional information divided for each tile and / or slice is generated.

[0757] At this time, the three-dimensional data encoding device determines reference values ​​and difference information of QP values ​​for each divided tile and / or slice (S5331). Specifically, the three-dimensional data encoding device determines reference values ​​and difference information as exemplified in FIGS. 104 to 106.

[0758] Then, the three-dimensional data encoding device updates additional information to include the determined reference value and difference information (S5332).

[0759] FIG. 109 is a flowchart showing an example of a process for encoding a determined QP value in the process of encoding (S5326).

[0760] The three-dimensional data encoding device encodes the QP value determined in step S5331 (S5341). Specifically, the three-dimensional data encoding device encodes the reference value and difference information of the QP value included in the updated additional information.

[0761] And, the three-dimensional data encoding device continues the encoding process until the stopping condition of the encoding process is satisfied, for example, until the data to be encoded is lost (S5342).

[0762] FIG. 110 is a flowchart of the decoding process of point cloud data according to the present embodiment. First, the three-dimensional data decoding device determines a division method by interpreting additional information related to the division method (tile additional information, position slice additional information and attribute slice additional information) included in the encoded data (encoded stream) (S5351). This division method includes whether to perform tile division or slice division. In addition, the division method may include the number of divisions when performing tile division or slice division, and the type of division, etc.

[0763] Next, the three-dimensional data decoding device generates segmentation position information and segmentation attribute information by decoding a plurality of encoding position information and a plurality of encoding attribute information included in the encoded data using dependency relationship information included in the encoded data (S5352).

[0764] When it is indicated that slice division is performed based on additional information (Yes in S5353), the three-dimensional data decoding device generates multiple tile location information and multiple tile attribute information by combining multiple division location information and multiple division attribute information based on location slice additional information and attribute slice additional information (S5354).

[0765] When it is indicated that tile division is performed based on additional information (Yes in S5355), the three-dimensional data decoding device generates position information and attribute information by combining a plurality of tile position information and a plurality of tile attribute information (a plurality of division position information and a plurality of division attribute information) based on the tile additional information (S5356).

[0766] FIG. 111 is a flowchart illustrating an example of a process for obtaining a QP value and decoding the QP value of a slice or tile in the combination of information divided per slice (S5354) or the combination of information divided per tile (S5356).

[0767] The position information and attribute information of slices or tiles may be combined using their respective methods, or combined using the same method.

[0768] The three-dimensional data decoding device decodes reference value and difference information from additional information of the encoding stream (S5361).

[0769] Next, the three-dimensional data decoding device calculates a quantization value using the decoded reference value and difference information, and updates the QP value used for inverse quantization with the calculated QP value (S5362). As a result, a QP value for inverse quantization of quantization attribute information for each tile or each slice can be derived.

[0770] And, the three-dimensional data decoding device continues the decoding process until the condition for stopping the decoding process is satisfied, for example, until the data to be decoded is lost (S5363).

[0771] FIG. 112 is a diagram showing an example of GPS syntax. FIG. 113 is a diagram showing an example of APS syntax. FIG. 114 is a diagram showing an example of location information header syntax. FIG. 115 is a diagram showing an example of attribute information header syntax.

[0772] As shown in FIG. 112, for example, GPS, which is supplementary information for location information, includes a QP_value representing an absolute value that serves as the basis for deriving the QP value. The QP value is, for example, the Q exemplified in FIG. 104 to 106. G It is equivalent to.

[0773] Also, as shown in FIG. 113, for example, the additional information of the attribute information, APS, may define a default time point when there is multiple color information of multiple time points at a three-dimensional point, and the 0th one must contain information of the default time point. For example, when a three-dimensional data encoding device decodes or displays a single color information, it may decode or display the 0th attribute information.

[0774] APS includes QP_delta_Attribute_to_Geometry. QP_delta_Attribute_to_Geometry represents difference information with respect to the reference value (QP_value) recorded in the GPS. This difference information is, for example, difference information with respect to luminance when the attribute information is color information.

[0775] In addition, the GPS may include a flag in the Geometry_header (header of position information) indicating whether there is difference information for calculating the QP value. In addition, the APS may include a flag in the Attribute_header (header of attribute information) indicating whether there is difference information for calculating the QP value. The flag may indicate whether there is difference information from the frame QP of the data QP for calculating the data QP in the attribute information.

[0776] In this way, the encoding stream may include identification information (flags) indicating that when a first color among the attribute information appears as a first luminance and a first color difference, quantization using a second quantization parameter for quantizing the first luminance, and quantization using a third quantization parameter for quantizing the first color difference, quantization using a fifth quantization parameter and a sixth quantization parameter, quantization using a fifth quantization parameter and a sixth quantization parameter.

[0777] Also, as shown in FIG. 114, the header of the location information may include QP_delta_data_to_frame, which represents the difference information with respect to the reference value (QP_value) recorded in the GPS. Additionally, the header of the location information may be divided into information for each tile and / or each slice, and for each tile and / or slice, the corresponding QP value may be displayed.

[0778] Also, as shown in FIG. 115, the header of the attribute information may include QP_delta_data_to_frame, which represents the difference information with the QP value listed in APS.

[0779] In FIGS. 104 to 106, the reference value of the QP value was described as the QP value of the position information in the PCC frame, but it is not limited to this, and other values ​​may be used as the reference value.

[0780] FIG. 116 is a diagram illustrating another example of a method for transmitting quantization parameters.

[0781] FIG. 116 (a) and (b) illustrate a fourth example of setting a common reference value Q from the QP values ​​of location information and attribute information in a PCC frame. In the fourth example, the reference value Q is stored in the GPS, and the QP value of the location information (Q) from the reference value Q is G Store the difference information of ) in the GPS, and the QP value (Q of the attribute information Y and Q R The difference information of ) is stored in APS. Also, the reference value Q may be stored in SPS.

[0782] Figures 116 (c) and (d) illustrate a fifth example in which reference values ​​are set independently for location information and attribute information. In this fifth example, the QP values ​​(absolute values) of the references for location information and attribute information are stored in the GPS and APS, respectively. That is, for the location information, the reference value Q G is set, and in the color information of the attribute information, the reference value Q Y is set, and for the reflectance of the attribute information, the reference value Q R These are each set. In this way, a reference value of the QP value may be set for each of the location information and multiple types of attribute information. Furthermore, the fifth example may be combined with other examples. That is, Q in the first example A , Q in the second example Y1 , Q Y2 , Q R It may be the reference value of the QP value.

[0783] FIG. 116 (e) and (f) illustrate a sixth example of setting a common reference value Q across multiple PCC frames when there are multiple PCC frames. In this sixth example, the reference value Q is stored in the SPS or GSPS, and the difference information between the QP value of the position information of each PCC frame and the reference value is stored in the GPS. Additionally, within the range of a random access unit, such as in a GOF, for example, the first frame of the random access unit is used as the reference value, and the difference information between PCC frames (e.g., Δ(Q) G(1) , Q G(0) It is permissible to transmit ))

[0784] In addition, even if a tile or slice is further subdivided, the difference information with the QP value of the subdivision unit is stored in the data header in the same way and transmitted.

[0785] FIG. 117 is a diagram illustrating another example of a method for transmitting quantization parameters.

[0786] Figures 117 (a) and (b) show a common reference value Q in the position information and attribute information in the PCC frame. G This shows the seventh example of setting. In the seventh example, the reference value Q G It stores in the GPS, and stores the difference information with location information or attribute information in their respective data headers. Reference value Q G It may be saved in SPS.

[0787] Also, FIG. 117 (c) and (d) illustrate an eighth example in which the QP value of attribute information is represented as difference information with respect to the QP value of location information belonging to the same slice and tile. In the eighth example, the reference value Q G ...may be saved in SPS.

[0788] FIG. 118 is a diagram for explaining the ninth example of a method for transmitting quantization parameters.

[0789] Figures 118 (a) and (b) are the ninth example in which a common QP value in the attribute information is interpreted to show the difference information with the QP value of the position information and the difference information with the common QP value in the attribute information, respectively.

[0790] FIG. 119 is a diagram illustrating an example of controlling the QP value.

[0791] As the value of the quantization parameter decreases, quality improves, but encoding efficiency decreases because more bits are required.

[0792] For example, when dividing three-dimensional point cloud data into tiles for encoding, if the point cloud data included in a tile represents a major road, it is encoded using the QP value of predefined attribute information. On the other hand, since surrounding tiles do not contain important information, there is a possibility of improving encoding efficiency by degrading the data quality through setting the difference information of the QP value to a positive value.

[0793] In addition, when encoding tile-divided three-dimensional point cloud data by dividing it into slices, sidewalks, trees, and buildings are set to negative QP values ​​because they are important for position estimation (localization and mapping) in autonomous driving, while moving objects and others are set to positive QP values ​​because they are of low importance.

[0794] FIG. 119(b) illustrates an example of deriving difference information when a quantization delta value is set in advance based on an object included in a tile or slice. For example, when the segmented data is slice data of a "building" included in a tile that is a "main road," the quantization delta value of the tile that is a "main road" is 0 and the quantization delta value of the slice data that is a "building" is -5 are added to derive the difference information as -5.

[0795] FIG. 120 is a flowchart showing an example of a method for determining a QP value based on the quality of an object.

[0796] The three-dimensional data encoding device divides point cloud data into one or more tiles based on map information and determines an object included in each of the one or more tiles (S5371). Specifically, the three-dimensional data encoding device performs object recognition processing to recognize what an object is, for example, by using a learning model obtained from machine learning.

[0797] Next, the three-dimensional data encoding device determines whether to encode the tiles to be processed with high quality (S5372). Encoding with high quality means, for example, encoding at a bit rate greater than a predetermined rate.

[0798] Next, when the three-dimensional data encoding device encodes the tile to be processed with high quality (Yes in S5372), it sets the QP value of the tile to increase the encoding quality (S5373).

[0799] Meanwhile, if the three-dimensional data encoding device does not encode the tile to be processed with high quality (No in S5372), it sets the QP value of the tile so that the encoding quality is lowered (S5374).

[0800] After step S5373 or step S5374, the three-dimensional data encoding device determines an object within a tile and divides it into one or more slices (S5375).

[0801] Next, the three-dimensional data encoding device determines whether to encode the slice of the processing target with high quality (S5376).

[0802] Next, when the three-dimensional data encoding device encodes the slice of the processing target with high quality (Yes in S5376), it sets the QP value of the slice to increase the encoding quality (S5377).

[0803] Meanwhile, if the three-dimensional data encoding device does not encode the slice of the processing target with high quality (No in S5376), it sets the QP value of the slice so that the encoding quality is lowered (S5378).

[0804] Next, the three-dimensional data encoding device determines reference values ​​and difference information to be transmitted in a predetermined manner based on a set QP value, and stores the determined reference values ​​and difference information in at least one of the headers of additional information and data (S5379).

[0805] Next, the three-dimensional data encoding device quantizes and encodes position information and attribute information based on the determined QP value (S5380).

[0806] FIG. 121 is a flowchart showing an example of a method for determining a QP value based on rate control.

[0807] The three-dimensional data encoding device encodes point cloud data sequentially (S5381).

[0808] Next, the three-dimensional data encoding device determines the rate control status according to the encoding process from the amount of encoding data and the amount of the encoding buff...

Claims

Claim 1 A method for encoding three-dimensional data, wherein a plurality of attribute information having each of a plurality of three-dimensional points is encoded using parameters, and a bit stream including the encoded plurality of attribute information, control information, and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a different type of attribute information, and the plurality of first attribute control information each corresponds to the plurality of attribute information, and each of the plurality of first attribute control information includes a first identification information indicating that it corresponds to any one of the plurality of type information. Claim 2 A three-dimensional data encoding method according to claim 1, wherein the plurality of type information is stored in a predetermined order in the control information, and the first identification information indicates that the first attribute control information including the first identification information corresponds to the type information of one of the predetermined order numbers. Claim 3 A three-dimensional data encoding method according to claim 1 or claim 2, wherein the bit stream further comprises a plurality of second attribute control information corresponding to the plurality of attribute information, and each of the plurality of second attribute control information comprises a reference value of a parameter used for encoding the corresponding attribute information. Claim 4 A three-dimensional data encoding method according to claim 3, wherein the first attribute control information comprises difference information which is the difference from the reference value of the parameter. Claim 5 A three-dimensional data encoding method according to claim 1 or claim 2, wherein the bit stream further comprises a plurality of second attribute control information corresponding to the plurality of attribute information, and each of the plurality of second attribute control information has second identification information indicating that it corresponds to any one of the plurality of type information. Claim 6 A three-dimensional data encoding method according to claim 1 or claim 2, wherein each of the plurality of first attribute control information has N fields in which N parameters (N is 2 or more) are stored, and in a specific first attribute control information corresponding to a specific type of attribute among the plurality of first attribute control information, one of the N fields includes a value indicating that it is invalid. Claim 7 A three-dimensional data encoding method according to claim 1 or claim 2, wherein the encoding quantizes the plurality of attribute information using a quantization parameter as the parameter. Claim 8 A method for decoding three-dimensional data, comprising acquiring a plurality of encoded attribute information and a parameter by acquiring a bit stream, and decoding the plurality of encoded attribute information using the parameter to decode the plurality of attribute information each having a plurality of three-dimensional points, wherein the bit stream includes control information and a plurality of first attribute control information, wherein the control information corresponds to the plurality of attribute information and includes a plurality of type information each indicating a different type of attribute information, wherein the plurality of first attribute control information each corresponds to the plurality of attribute information, and each of the plurality of first attribute control information includes a first identification information indicating that it corresponds to any one of the plurality of type information. Claim 9 A three-dimensional data decoding method according to claim 8, wherein the plurality of type information is stored in a predetermined order in the control information, and the first identification information indicates that the first attribute control information including the first identification information corresponds to the type information of one of the predetermined order numbers. Claim 10 A three-dimensional data decoding method according to claim 8 or claim 9, wherein the bit stream further comprises a plurality of second attribute control information corresponding to the plurality of attribute information, and each of the plurality of second attribute control information comprises a reference value of a parameter used for decoding the corresponding attribute information. Claim 11 A three-dimensional data decoding method according to claim 10, wherein the first attribute control information comprises difference information which is the difference from the reference value of the parameter. Claim 12 A three-dimensional data decoding method according to claim 8 or claim 9, wherein the bit stream further comprises a plurality of second attribute control information corresponding to the plurality of attribute information, and each of the plurality of second attribute control information has second identification information indicating that it corresponds to any one of the plurality of type information. Claim 13 A three-dimensional data decoding method according to claim 8 or claim 9, wherein each of the plurality of first attribute control information has a plurality of fields in which a plurality of parameters are stored, and in the decoding, a parameter stored in a specific field among the plurality of fields of specific first attribute control information corresponding to a specific type of attribute among the plurality of first attribute control information is ignored. Claim 14 A three-dimensional data decoding method according to claim 8 or claim 9, wherein in the decoding, the encoded plurality of attribute information is inversely quantized using a quantization parameter as the parameter. Claim 15 A three-dimensional data encoding device comprising a processor and a memory, wherein the processor uses the memory to encode a plurality of attribute information having each of a plurality of three-dimensional points using parameters, and generates a bit stream including the encoded plurality of attribute information, control information, and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a different type of attribute information, and the plurality of first attribute control information includes (i) each of the plurality of attribute information and (ii) a first identification information indicating that it corresponds to one of the plurality of type information. Claim 16 A three-dimensional data decoding device comprising a processor and a memory, wherein the processor acquires a plurality of encoded attribute information and a parameter by acquiring a bit stream using the memory, and decodes a plurality of attribute information having each of a plurality of three-dimensional points by decoding the encoded plurality of attribute information using the parameter, wherein the bit stream includes control information and a plurality of first attribute control information, wherein the control information includes a plurality of type information corresponding to the plurality of attribute information and each of which indicates a type of attribute information different from each other, and wherein the plurality of first attribute control information includes (i) each corresponding to the plurality of attribute information and (ii) a first identification information indicating that it corresponds to any one of the plurality of type information. Claim 17 A method for encoding a point cloud comprising one or more attributes corresponding to a point, wherein the method encodes the one or more attributes and generates the encoded data comprising a first parameter set for the encoded data and one or more attribute data units, wherein each of the one or more attribute data units comprises an attribute header, wherein the first parameter set comprises one or more attribute parameters, wherein the one or more attribute parameters correspond one-to-one with the one or more attributes, wherein the encoded data comprises one or more second parameter sets each comprising an identifier, and wherein the attribute header comprises an identifier of a corresponding second parameter set among the one or more second parameter sets. Claim 18 A point cloud encoding method according to claim 17, wherein the attribute header includes a value indicating the sequence number of an attribute parameter corresponding to one of the attribute data units among one or more attribute parameters stored in the first parameter set. Claim 19 A point cloud encoding method according to claim 17, wherein one or more attributes include a first attribute representing the color of the point, and the first attribute includes three components. Claim 20 delete Claim 21 A point cloud encoding method according to claim 17, wherein each of the one or more second parameter sets includes a first quantization parameter value, and the attribute header includes a second quantization parameter value which is an offset between the quantization parameter used for decoding and the first quantization parameter value. Claim 22 A point cloud encoder for encoding a point cloud comprising one or more attributes corresponding to a point, comprising a processor and a memory, wherein the processor uses the memory to encode the one or more attributes and generates the encoded data comprising a first parameter set for the encoded data and one or more attribute data units, wherein each of the one or more attribute data units comprises an attribute header, wherein the first parameter set comprises one or more attribute parameters, wherein the one or more attribute parameters correspond one-to-one with the one or more attributes, wherein the encoded data comprises one or more second parameter sets each comprising an identifier, and wherein the attribute header comprises an identifier of a corresponding second parameter set among the one or more second parameter sets. Claim 23 A point cloud encoder according to claim 22, wherein the attribute header includes a value indicating the sequence number of an attribute parameter corresponding to an attribute data unit of one or more attribute parameters stored in the first parameter set. Claim 24 A point cloud encoder according to claim 22, wherein one or more attributes include a first attribute representing the color of the point, and the first attribute includes three components. Claim 25 delete Claim 26 A point cloud encoder according to claim 22, wherein each of the one or more second parameter sets includes a first quantization parameter value, and the attribute header includes a second quantization parameter value which is an offset between the quantization parameter used for decoding and the first quantization parameter value. Claim 27 A decoding method for decoding encoded data to obtain a point cloud comprising one or more attributes corresponding to a point, wherein the method comprises obtaining a first parameter set and one or more attribute data units for the encoded data from the encoded data, decoding the one or more attribute data units based on the first parameter set, wherein each of the one or more attribute data units comprises an attribute header, wherein the first parameter set comprises one or more attribute parameters, wherein the one or more attribute parameters correspond one-to-one with the one or more attributes, wherein the encoded data comprises one or more second parameter sets each comprising an identifier, and wherein the attribute header comprises an identifier of a corresponding second parameter set among the one or more second parameter sets. Claim 28 A method for decoding encoded data according to claim 27, wherein the attribute header includes a value indicating the sequence number of an attribute parameter corresponding to one of the attribute data units among one or more attribute parameters stored in the first parameter set. Claim 29 A method for decoding encoded data according to claim 27, wherein one or more attributes include a first attribute representing the color of the point, and the first attribute includes three components. Claim 30 delete Claim 31 A method for decoding encoded data according to claim 27, wherein each of the one or more second parameter sets includes a first quantization parameter value, and the attribute header includes a second quantization parameter value which is an offset between the quantization parameter used for decoding and the first quantization parameter value. Claim 32 A decoder for decoding encoded data to obtain a point cloud including one or more attributes corresponding to a point, comprising a processor and a memory, wherein the processor uses the memory to obtain a first parameter set and one or more attribute data units for the encoded data from the encoded data, and decodes the one or more attribute data units based on the first parameter set, wherein each of the one or more attribute data units includes an attribute header, wherein the first parameter set includes one or more attribute parameters, wherein the one or more attribute parameters correspond one-to-one with the one or more attributes, wherein the encoded data includes one or more second parameter sets each including an identifier, and wherein the attribute header includes an identifier of a corresponding second parameter set among the one or more second parameter sets. Claim 33 An encoded data decoder according to claim 32, wherein the attribute header includes a value indicating the sequence number of an attribute parameter corresponding to an attribute data unit of one or more attribute parameters stored in the first parameter set. Claim 34 An encoded data decoder according to claim 32, wherein one or more attributes include a first attribute representing the color of the point, and the first attribute includes three components. Claim 35 delete Claim 36 An encoded data decoder according to claim 32, wherein each of the one or more second parameter sets includes a first quantization parameter value, and the attribute header includes a second quantization parameter value which is an offset between the quantization parameter used for decoding and the first quantization parameter value.

Images