Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

Abstract

A three-dimensional data encoding method for encoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of the plurality of first components is expressed by one or more dimensions. The three-dimensional data encoding method selects any one of the plurality of encoding methods (S321), generates encoded data by encoding the plurality of first components using the selected encoding method (S322), and generates a bit stream including the encoded data and first information indicating the selected encoding method (S323).

Description

Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

[0001] The present disclosure relates to a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, and a three-dimensional data decoding device.

[0002] In the future, devices and services that utilize 3D data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, video distribution, etc. 3D data can be acquired in a variety of ways, including distance sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

[0003] One method of representing three-dimensional data is a representation method called a point cloud, which represents the shape of a three-dimensional structure using a group of points in three-dimensional space. In a point cloud, the positions and colors of the points are stored. Point clouds are expected to become mainstream as a method of representing three-dimensional data, but point clouds have a very large amount of data. Therefore, when storing or transmitting three-dimensional data, it is essential to compress the amount of data by encoding, just as with two-dimensional moving images (examples include MPEG-4 AVC (Advanced Video Coding) or HEVC (High Efficiency Video Coding) standardized by MPEG).

[0004] Furthermore, compression of point clouds is partially supported by a public library (Point Cloud Library) that performs processing related to point clouds.

[0005] Furthermore, a technique is known in which three-dimensional map data is used to search for and display facilities located around a vehicle (see, for example, Patent Document 1).

[0006] International Publication No. 2014 / 020663

[0007] In encoding and decoding such three-dimensional data, it is desirable to be able to improve the encoding efficiency.

[0008] An object of the present disclosure is to provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency.

[0009] A three-dimensional data encoding method according to one aspect of the present disclosure is a three-dimensional data encoding method for encoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of which is represented in one or more dimensions, and the three-dimensional data encoding method selects one of a plurality of encoding methods, generates encoded data by encoding the plurality of first components using the selected encoding method, and generates a bitstream including the encoded data and first information indicating the selected encoding method.

[0010] A three-dimensional data decoding method according to one aspect of the present disclosure is a three-dimensional data decoding method for decoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of which is represented in one or more dimensions; the three-dimensional data decoding method includes: acquiring a bit stream including encoded data generated by encoding the plurality of first components; acquiring first information indicating an encoding method used to encode the plurality of first components from among a plurality of encoding methods included in the bit stream; and generating the plurality of first components by decoding the encoded data using a decoding method corresponding to the encoding method indicated by the first information.

[0011] The present disclosure can provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency.

[0012] FIG. 1 is a diagram illustrating an example of the configuration of a 3D data encoding / decoding system according to an embodiment. FIG. 2 is a diagram illustrating an example of the configuration of point cloud data according to an embodiment. FIG. 3 is a diagram illustrating an example of the configuration of a data file in which point cloud data information according to an embodiment is described. FIG. 4 is a diagram illustrating an example of the configuration of mesh data according to an embodiment. FIG. 5 is a diagram illustrating an example of the configuration of a data file in which mesh data information according to an embodiment is described. FIG. 6 is a diagram illustrating types of 3D data according to an embodiment. FIG. 7 is a diagram illustrating the configuration of a first encoding unit according to an embodiment. FIG. 8 is a block diagram of a first encoding unit according to an embodiment. FIG. 9 is a diagram illustrating the configuration of a first decoding unit according to an embodiment. FIG. 10 is a block diagram of a first decoding unit according to an embodiment. FIG. 11 is a diagram illustrating the configuration of a second encoding unit according to an embodiment. FIG. 12 is a block diagram of a second encoding unit according to an embodiment. FIG. 13 is a diagram illustrating the configuration of a second decoding unit according to an embodiment. FIG. 14 is a block diagram of a second decoding unit according to an embodiment. FIG. 15 is a block diagram of a position information encoding unit according to an embodiment. FIG. 16 is a block diagram of a position information decoding unit according to an embodiment. FIG. 17 is a block diagram of an octree encoding unit according to an embodiment. FIG. 18 is a diagram showing an example of position information according to an embodiment. FIG. 19 is a diagram showing an example of an octree representation of position information according to an embodiment. FIG. 20 is a block diagram of an octree decoding unit according to an embodiment. FIG. 21 is a block diagram of an attribute information encoding unit according to an embodiment. FIG. 22 is a block diagram of an attribute information decoding unit according to an embodiment. FIG. 23 is a block diagram of an attribute information encoding unit according to an embodiment. FIG. 24 is a block diagram of an attribute information decoding unit according to an embodiment. FIG. 25 is a block diagram of a 3D data encoding device according to an embodiment. FIG. 26 is a block diagram of a 3D data decoding device according to an embodiment. FIG. 27 is a diagram showing the relationship between tiles and slices according to an embodiment. FIG. 28 is a diagram showing an example configuration of a bitstream according to an embodiment. FIG. 29 is a block diagram showing an example configuration of a 3D data encoding device according to an embodiment. FIG. 30 is a diagram showing example configurations of encoded data and NAL units according to an embodiment.FIG. 31 is a diagram showing example semantics of pcc_nal_unit_type according to an embodiment. FIG. 32 is a diagram showing part of processing in a 3D data generation system according to an embodiment. FIG. 33 is a diagram showing components of Gaussian data according to an embodiment. FIG. 34 is a diagram showing rendering processing of Gaussian data according to an embodiment. FIG. 35 is a diagram showing an example of the number of SH coefficients of a spherical harmonic function according to an embodiment. FIG. 36 is a diagram showing example input / output of a spherical harmonic function according to an embodiment. FIG. 37 is a diagram showing rendering processing according to an embodiment. FIG. 38 is a block diagram of an encoding device according to an embodiment. FIG. 39 is a block diagram of a decoding device according to an embodiment. FIG. 40 is a block diagram of an encoding device according to an embodiment. FIG. 41 is a block diagram of a decoding device according to an embodiment. FIG. 42 is a diagram showing example mapping of Gaussian data according to an embodiment. FIG. 43 is a diagram showing example mapping information according to an embodiment. FIG. 44 is a diagram showing an example configuration of conversion information according to an embodiment. FIG. 45 is a diagram showing an example configuration of encoded data according to an embodiment. FIG. 46 is a diagram showing an example G-PCC attribute type of Gaussian data according to an embodiment. FIG. 47 is a flowchart of processing by a decoding device according to an embodiment. FIG. 48 is a block diagram showing a system for encoding and decoding first Gaussian data according to an embodiment. FIG. 49 is a block diagram of a second pre-processing unit according to an embodiment. FIG. 50 is a diagram showing conversion processing according to an embodiment. FIG. 51 is a diagram showing a protocol stack of an encoding standard for encoding Gaussian data according to an embodiment. FIG. 52 is a block diagram of an encoding device according to an embodiment. FIG. 53 is a block diagram of a decoding device according to an embodiment. FIG. 54 is a flowchart of encoding processing according to an embodiment. FIG. 55 is a flowchart of decoding processing according to an embodiment. FIG. 56 is a diagram showing a basic data structure of a Gaussian codec according to an embodiment. FIG. 57 is a diagram showing an example of the structure of a 3D data unit according to an embodiment. FIG. 58 is a diagram showing an example of the structure of a 3DPS according to an embodiment. FIG. 59 is a diagram showing an example of the structure of Gaussian data component information according to an embodiment.FIG. 60 is a diagram showing an example of the configuration of G-PCC mapping information according to an embodiment. FIG. 61 is a diagram showing an example of the configuration of V3C mapping information according to an embodiment. FIG. 62 is a diagram showing an example of the configuration of a Gaussian data unit according to an embodiment. FIG. 63 is a diagram showing a protocol stack of a standard for encoding a 3D Gaussian according to an embodiment. FIG. 64 is a diagram showing a protocol stack of a standard for encoding a 3D model according to an embodiment. FIG. 65 is a diagram showing an example of the configuration of Gaussian component information according to an embodiment. FIG. 66 is a flowchart of encoding processing by an encoding device according to an embodiment. FIG. 67 is a block diagram of an encoding device according to an embodiment. FIG. 68 is a flowchart of decoding processing by a decoding device according to an embodiment. FIG. 69 is a block diagram of a decoding device according to an embodiment. FIG. 70 is a flowchart of encoding processing by an encoding device according to an embodiment. FIG. 71 is a flowchart of decoding processing by a decoding device according to an embodiment.

[0013] [Introduction] A three-dimensional data encoding method according to one aspect of the present disclosure is a three-dimensional data encoding method for encoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of which is expressed in one or more dimensions, and the plurality of first components includes a first component that indicates position coordinates and a first component that indicates information about a scale value based on the position coordinates, and the three-dimensional data encoding method generates encoded data by encoding the plurality of first components using a point cloud encoding method, and generates a bitstream that includes the encoded data.

[0014] According to this, the three-dimensional data encoding method can encode Gaussian data using a point cloud encoding method, which allows Gaussian data to be encoded without creating a new encoding method for Gaussian data, thereby achieving efficient processing.

[0015] For example, the plurality of first components may further include a first component indicating information on a rotation value based on the position coordinates, and may further include a first component indicating color information or a first component indicating transmittance.

[0016] For example, the rotation and scale values ​​may be represented as information in a covariance matrix.For example, the plurality of first components may further include a first component representing a coefficient of a spherical harmonic function.

[0017] For example, the three-dimensional data encoding method may further convert the plurality of first components into a plurality of second components, and encode the plurality of second components in the encoding. According to this, for example, the three-dimensional data encoding method can encode Gaussian data using a point cloud encoding method by converting data into a format compatible with the point cloud encoding method.

[0018] For example, the three-dimensional data encoding method may further include storing information indicating a correspondence between the plurality of first components and the plurality of second components in metadata, and the bitstream may further include the metadata, thereby allowing a decoding device to convert the plurality of second components into the plurality of first components using the information included in the bitstream.

[0019] For example, the number of dimensions of the plurality of second components may be equal to or less than a predetermined number, and in the conversion, a first component of the plurality of first components having a number of dimensions greater than the predetermined number may be divided into two or more second components. According to this, for example, the three-dimensional data encoding method can encode Gaussian data using a point cloud encoding method by converting data into a format compatible with the point cloud encoding method.

[0020] For example, in the encoding, the first component indicating the position coordinates may be encoded using a geometry encoding method included in the point cloud encoding method, and the first component indicating the scale value information may be encoded using an attribute encoding method included in the point cloud encoding method. In this way, the three-dimensional data encoding method can appropriately encode each piece of information using the geometry encoding method and the attribute encoding method included in the point cloud encoding method.

[0021] For example, the bitstream may include a parameter set including information indicating the types of the first components, thereby allowing the decoding device to determine the types of the first components using the information included in the bitstream.

[0022] A three-dimensional data decoding method according to one aspect of the present disclosure is a three-dimensional data decoding method for decoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of which is expressed in one or more dimensions, and the plurality of first components include a first component indicating a position coordinate and a first component indicating information on a scale value based on the position coordinate, and the three-dimensional data decoding method obtains a bit stream including encoded data generated by encoding the plurality of first components, and generates the plurality of first components by decoding the encoded data using a point cloud decoding method.

[0023] According to this, the encoding device can encode Gaussian data using a point cloud encoding method. Therefore, Gaussian data can be encoded without creating a new encoding method for Gaussian data, thereby achieving efficient processing. Furthermore, the three-dimensional data decoding method can appropriately decode encoded data generated by the encoding device.

[0024] For example, the plurality of first components may further include a first component indicating information on a rotation value based on the position coordinates, and may further include a first component indicating color information or a first component indicating transmittance.

[0025] For example, the rotation and scale values ​​may be represented as information in a covariance matrix.For example, the plurality of first components may further include a first component representing a coefficient of a spherical harmonic function.

[0026] For example, the encoded data may be generated by encoding a plurality of second components into which the plurality of first components are converted, and in the decoding, the encoded data may be decoded using a point cloud decoding method to generate the plurality of second components, and the plurality of second components may be converted into the plurality of first components.

[0027] According to this, for example, the encoding device can generate encoded data by encoding Gaussian data using the point cloud encoding method by converting the data into a format compatible with the point cloud encoding method. Also, the three-dimensional data decoding method can appropriately decode the encoded data generated by the encoding device.

[0028] For example, the three-dimensional data decoding method may further acquire information indicating a correspondence between the plurality of first components and the plurality of second components from metadata included in the bitstream, thereby enabling the three-dimensional data decoding method to convert the plurality of second components into the plurality of first components using the information included in the bitstream.

[0029] For example, the number of dimensions of the plurality of second components may be less than or equal to a predetermined number, and the transformation may combine two or more of the plurality of second components into a first component having a number of dimensions greater than the predetermined number.

[0030] According to this, for example, the encoding device can encode Gaussian data using the point cloud encoding method by converting the data into a format compatible with the point cloud encoding method. Also, the three-dimensional data decoding method can appropriately decode the encoded data generated by the encoding device.

[0031] For example, in the decoding, the first component indicating the position coordinates may be decoded using a geometry decoding method included in the point cloud decoding method, and the first component indicating the scale value information may be decoded using an attribute decoding method included in the point cloud decoding method.

[0032] This allows the encoding device to appropriately encode each piece of information using the geometry encoding method and attribute encoding method included in the point cloud encoding method, and the decoding device to appropriately decode the encoded data generated by the encoding device.

[0033] For example, the bitstream may include a parameter set including information indicating the types of the plurality of first components, whereby the three-dimensional data decoding method can determine the types of the plurality of first components using the information included in the bitstream.

[0034] A three-dimensional data encoding device according to one aspect of the present disclosure is a three-dimensional data encoding device that encodes Gaussian data, comprising a processor and a memory, wherein the Gaussian data includes a plurality of first components, each of which is expressed in one or more dimensions, and the plurality of first components includes a first component that indicates position coordinates and a first component that indicates information on a scale value based on the position coordinates, and the processor uses the memory to generate encoded data by encoding the plurality of first components using a point cloud encoding method, and generates a bitstream that includes the encoded data.

[0035] This allows the three-dimensional data encoding device to encode Gaussian data using a point cloud encoding method, thereby enabling efficient processing since Gaussian data can be encoded without creating a new encoding method for Gaussian data.

[0036] A three-dimensional data decoding device according to one aspect of the present disclosure is a three-dimensional data decoding device that decodes Gaussian data, and is equipped with a processor and a memory, wherein the Gaussian data includes a plurality of first components, each of which is expressed in one or more dimensions, and the plurality of first components include a first component that indicates a position coordinate and a first component that indicates information about a scale value based on the position coordinate, and the processor uses the memory to obtain a bit stream including encoded data generated by encoding the plurality of first components, and generates the plurality of first components by decoding the encoded data using a point cloud decoding method.

[0037] According to this, the encoding device can encode Gaussian data using a point cloud encoding method. Therefore, Gaussian data can be encoded without creating a new encoding method for Gaussian data, thereby achieving efficient processing. Furthermore, the three-dimensional data decoding device can appropriately decode the encoded data generated by the encoding device.

[0038] A three-dimensional data encoding method according to one aspect of the present disclosure is a three-dimensional data encoding method for encoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of which is represented in one or more dimensions, and the three-dimensional data encoding method selects one of a plurality of encoding methods, generates encoded data by encoding the plurality of first components using the selected encoding method, and generates a bitstream including the encoded data and first information indicating the selected encoding method.

[0039] This three-dimensional data encoding method allows selective selection of an encoding method to be used for encoding Gaussian data from among a plurality of encoding methods, which may improve encoding efficiency.

[0040] For example, the three-dimensional data encoding method may further convert the plurality of first components into a plurality of second components corresponding to the selected encoding method, and the encoding may encode the plurality of second components, and the bitstream may further include second information indicating a correspondence between the plurality of first components and the plurality of second components.

[0041] According to this, the three-dimensional data encoding method can encode Gaussian data using a selected encoding method by converting data into a format corresponding to the selected encoding method, and the decoding device can convert multiple second components into multiple first components using second information included in the bitstream.

[0042] For example, the plurality of first components may include a first component indicating a position coordinate and a first component indicating information on a scale value based on the position coordinate. For example, the plurality of encoding methods may include a geometry-based encoding method and a video-based encoding method.

[0043] For example, the plurality of encoding methods may further include an encoding method for a neural network. For example, the bitstream may include a parameter set common to the plurality of encoding methods, and the parameter set may include the first information.

[0044] For example, the bitstream may include a plurality of units common to the plurality of encoding methods, the first information may include a plurality of pieces of third information included in the header of each of the plurality of units, each of the plurality of units may store either first encoded data encoded using the geometry-based encoding method, second encoded data encoded using the video-based encoding method, or a plurality of pieces of data including the common parameters, and each of the plurality of pieces of third information may indicate the type of data stored in the unit in which the third information is included.

[0045] A three-dimensional data decoding method according to one aspect of the present disclosure is a three-dimensional data decoding method for decoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of which is represented in one or more dimensions; the three-dimensional data decoding method includes: acquiring a bit stream including encoded data generated by encoding the plurality of first components; acquiring first information indicating an encoding method used to encode the plurality of first components from among a plurality of encoding methods included in the bit stream; and generating the plurality of first components by decoding the encoded data using a decoding method corresponding to the encoding method indicated by the first information.

[0046] This allows the encoding device to selectively select an encoding method to be used for encoding Gaussian data from among multiple encoding methods, which may improve encoding efficiency. Furthermore, the three-dimensional data decoding method allows appropriate decoding of encoded data generated by the encoding device.

[0047] For example, the encoded data may be generated by encoding a plurality of second components into which the plurality of first components are converted, and the three-dimensional data decoding method may further acquire second information indicating a correspondence between the plurality of first components and the plurality of second components from the bitstream, and in the decoding, generate the plurality of second components by decoding the encoded data using the decoding method, and convert the plurality of second components into the plurality of first components using the correspondence indicated by the second information.

[0048] According to this, the encoding device can encode the Gaussian data using the selected encoding method by converting the data into a format corresponding to the selected encoding method. Also, the three-dimensional data decoding method can convert multiple second components into multiple first components using second information included in the bitstream.

[0049] For example, the plurality of first components may include a first component indicating a position coordinate and a first component indicating information on a scale value based on the position coordinate. For example, the plurality of encoding methods may include a geometry-based encoding method and a video-based encoding method.

[0050] For example, the plurality of encoding methods may further include an encoding method for a neural network. For example, the bitstream may include a parameter set common to the plurality of encoding methods, and the parameter set may include the first information.

[0051] For example, the bitstream may include a plurality of units common to the plurality of encoding methods, the first information may include a plurality of pieces of third information included in the header of each of the plurality of units, each of the plurality of units may store either first encoded data encoded using the geometry-based encoding method, second encoded data encoded using the video-based encoding method, or a plurality of pieces of data including the common parameters, and each of the plurality of pieces of third information may indicate the type of data stored in the unit in which the third information is included.

[0052] A three-dimensional data encoding device according to one aspect of the present disclosure is a three-dimensional data encoding device that encodes Gaussian data, comprising a processor and a memory, wherein the Gaussian data includes a plurality of first components, each of which is represented in one or more dimensions, and the processor uses the memory to select one of a plurality of encoding methods, generates encoded data by encoding the plurality of first components using the selected encoding method, and generates a bitstream that includes the encoded data and first information indicating the selected encoding method.

[0053] This allows the three-dimensional data encoding device to selectively select an encoding method to be used for encoding Gaussian data from a plurality of encoding methods, which may improve encoding efficiency.

[0054] A three-dimensional data decoding device according to one aspect of the present disclosure is a three-dimensional data decoding device that decodes Gaussian data, the three-dimensional data decoding device comprising a processor and a memory, wherein the Gaussian data includes a plurality of first components, each of which is represented in one or more dimensions, and the processor uses the memory to obtain a bit stream including encoded data generated by encoding the plurality of first components, obtains first information indicating an encoding method used to encode the plurality of first components from among a plurality of encoding methods included in the bit stream, and generates the plurality of first components by decoding the encoded data using a decoding method corresponding to the encoding method indicated by the first information.

[0055] This allows the encoding device to selectively select an encoding method to be used for encoding Gaussian data from among multiple encoding methods, which may improve encoding efficiency. Also, the three-dimensional data decoding device can appropriately decode the encoded data generated by the encoding device.

[0056] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0057] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0058] (Embodiment) [Three-dimensional data encoding / decoding system] First, the configuration of a three-dimensional data encoding / decoding system according to this embodiment will be described. Fig. 1 is a diagram showing an example of the configuration of a three-dimensional data encoding / decoding system according to this embodiment. As shown in Fig. 1, the three-dimensional data encoding / decoding system includes a three-dimensional data encoding system 101, a three-dimensional data decoding system 102, a sensor terminal 103, and an external connection unit 104.

[0059] The three-dimensional data encoding system 101 generates encoded data or multiplexed data by encoding three-dimensional data such as three-dimensional point cloud data or three-dimensional mesh data. The three-dimensional data encoding system 101 may be a three-dimensional data encoding device realized by a single device, or may be a system realized by multiple devices. The three-dimensional data encoding device may also include some of the multiple processing units included in the three-dimensional data encoding system 101.

[0060] The three-dimensional data encoding system 101 includes a three-dimensional data generation system 111, a presentation unit 112, an encoding unit 113, a multiplexing unit 114, an input / output unit 115, and a control unit 116. The three-dimensional data generation system 111 includes a sensor information acquisition unit 117 and a three-dimensional data generation unit 118.

[0061] The sensor information acquisition unit 117 acquires sensor information (sensor signals) from the sensor terminal 103 and outputs the sensor information to the three-dimensional data generation unit 118. The three-dimensional data generation unit 118 generates three-dimensional data from the sensor information and outputs the three-dimensional data to the encoding unit 113.

[0062] The presentation unit 112 presents the sensor information or the three-dimensional data to the user. For example, the presentation unit 112 displays information or an image based on the sensor information or the three-dimensional data.

[0063] The encoding unit 113 encodes (compresses) the three-dimensional data and outputs the resulting encoded data, control information obtained in the encoding process, and other additional information to the multiplexing unit 114. The additional information includes, for example, sensor information.

[0064] The multiplexing unit 114 generates multiplexed data by multiplexing the coded data, control information, and additional information input from the coding unit 113. The format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.

[0065] The input / output unit 115 (e.g., a communication unit or an interface) outputs the multiplexed data to the outside. Alternatively, the multiplexed data is stored in a storage unit such as an internal memory. The control unit 116 (or application execution unit) controls each processing unit. In other words, the control unit 116 controls encoding, multiplexing, etc.

[0066] The sensor information may be input to the encoding unit 113 or the multiplexing unit 114. The input / output unit 115 may output the three-dimensional data or the encoded data directly to the outside.

[0067] The transmission signal (multiplexed data) output from the three-dimensional data encoding system 101 is input to the three-dimensional data decoding system 102 via the external connection unit 104 .

[0068] The three-dimensional data decoding system 102 generates three-dimensional data such as three-dimensional point cloud data or three-dimensional mesh data by decoding encoded data or multiplexed data. The three-dimensional data decoding system 102 may be a three-dimensional data decoding device realized by a single device, or may be a system realized by multiple devices. Furthermore, the three-dimensional data decoding device may include some of the multiple processing units included in the three-dimensional data decoding system 102.

[0069] The three-dimensional data decoding system 102 includes a sensor information acquisition unit 121 , an input / output unit 122 , a demultiplexing unit 123 , a decoding unit 124 , a presentation unit 125 , a user interface 126 , and a control unit 127 .

[0070] The sensor information acquisition unit 121 acquires sensor information (sensor signals) from the sensor terminal 103 .

[0071] The input / output unit 122 acquires a transmission signal, decodes multiplexed data (file format or packets) from the transmission signal, and outputs the multiplexed data to the demultiplexer 123 .

[0072] The demultiplexer 123 obtains the coded data, control information, and additional information from the multiplexed data, and outputs the coded data, control information, and additional information to the decoder 124 .

[0073] The decoding unit 124 reconstructs three-dimensional data by decoding the encoded data.

[0074] The presentation unit 125 presents the three-dimensional data to the user. For example, the presentation unit 125 displays information or an image based on the three-dimensional data. The user interface 126 acquires instructions based on user operations. The control unit 127 (or the application execution unit) controls each processing unit. That is, the control unit 127 controls demultiplexing, decoding, presentation, etc.

[0075] The input / output unit 122 may acquire the three-dimensional data or the encoded data directly from the outside. The presentation unit 125 may acquire additional information such as sensor information and present information based on the additional information. The presentation unit 125 may also present information based on a user instruction acquired by the user interface 126.

[0076] The sensor terminal 103 generates sensor information, which is information obtained by a sensor. The sensor terminal 103 is a terminal equipped with a sensor or a camera, and may be, for example, a mobile object such as an automobile, a flying object such as an airplane, a mobile terminal, or a camera.

[0077] Examples of sensor information that can be acquired by the sensor terminal 103 include (1) the distance between the sensor terminal 103 and an object or the reflectance of the object obtained from a LIDAR, millimeter-wave radar, or infrared sensor, and (2) the distance between a camera and an object or the reflectance of the object obtained from multiple monocular camera images or stereo camera images. The sensor information may also include the attitude, orientation, gyro (angular velocity), position (GPS information or altitude), speed, acceleration, etc. of the sensor. The sensor information may also include temperature, air pressure, humidity, magnetism, etc.

[0078] The external connection unit 104 is realized by an integrated circuit (LSI or IC), an external storage unit, communication with a cloud server via the Internet, broadcasting, or the like.

[0079] Next, three-dimensional point cloud data (hereinafter also referred to as point cloud data) will be described. Fig. 2 is a diagram showing the structure of point cloud data. Fig. 3 is a diagram showing an example of the structure of a data file in which information about point cloud data is written.

[0080] Point cloud data includes data on multiple points. Each point includes location information (three-dimensional coordinates) and attribute information for that location. A collection of multiple points is called a point cloud. For example, a point cloud can represent the three-dimensional shape of an object.

[0081] Position information such as three-dimensional coordinates is sometimes called geometry. Data for each point may include attribute information of multiple attribute types. The attribute types may be, for example, color or reflectance.

[0082] One piece of attribute information may be associated with one piece of location information, or multiple pieces of attribute information having different attribute types may be associated with one piece of location information, or multiple pieces of attribute information of the same attribute type may be associated with one piece of location information.

[0083] The data file configuration example shown in FIG. 3 is an example in which position information and attribute information correspond one-to-one, and shows the position information and attribute information of N points that make up the point cloud data.

[0084] The position information is, for example, information on three axes, x, y, and z. The attribute information is, for example, RGB color information. A typical data file is a ply file.

[0085] Next, three-dimensional mesh data (hereinafter also referred to as mesh data) will be described. Fig. 4 is a diagram showing an example of the structure of mesh data. Fig. 5 is a diagram showing an example of the structure of a data file in which information about the mesh data is written.

[0086] Mesh data is a data format used in CG (Computer Graphics). Mesh data represents the three-dimensional shape of an object using a collection of surface information. The surface information is a polygon such as a triangle or a quadrangle, and is also called a polygon or polygon mesh.

[0087] The components of mesh data are a three-dimensional point cloud (a set of points having three-dimensional position information and attribute information corresponding to that position information), as well as a set of three-dimensional points as vertices, edges connecting two vertices, and faces enclosed by the edges.

[0088] A vertex (also expressed as a "vertex" or "position") may have attribute information such as color information, reflectance, or normal vector for the point. Information indicating the relationship between the vertices that make up an edge or face is also called connectivity. The front and back of a face can be expressed by the direction of the normal vector for the point. Mesh data may also have attribute information for the face.

[0089] An example of the format of a mesh data file is an object file. The data file indicates the position information G(1) to G(N) of the N vertices that make up the mesh, and the attribute information A(1) to A(N) of the vertices. Note that the data file does not have to include the attribute information. Also, the attribute information does not have to correspond one-to-one to the vertices. Note that FIG. 5 shows an example in which the data file has 1 to M pieces of attribute information A2.

[0090] Face information is represented by a combination of vertex indices. n[1, 3, 4] indicates that the face is a triangular face consisting of three vertices, n=1, n=3, and n=4. Furthermore, m[2, 4, 6] indicates that the attribute information of m=1, m=4, and m=6 corresponds to the three vertices, respectively.

[0091] Alternatively, the attribute information may be stored in a file separate from the data file, with the data file indicating its pointer information. For example, the attribute information may be stored in a two-dimensional attribute map file, with the file name of the attribute map and two-dimensional coordinates in the attribute map being stored in attribute information A2 of the data file. Either method can be used to specify attribute information for a point.

[0092] Next, the types of three-dimensional data (point cloud data or mesh data) will be described. Fig. 6 is a diagram showing the types of three-dimensional data. As shown in Fig. 6, three-dimensional data includes static objects and dynamic objects.

[0093] A static object is three-dimensional data at any time (a certain time). A dynamic object is three-dimensional data that changes over time. Hereinafter, three-dimensional point cloud data at a certain time will be referred to as a PCC (Point Cloud Compression) frame or frame. Also, three-dimensional mesh data at a certain time will be referred to as a mesh frame or frame.

[0094] The multiple points that make up an object may be limited to a certain extent in terms of area, number of pixels, or number of points, as in normal video data, or the multiple points that make up an object may not be limited in terms of area, as in map information.

[0095] Furthermore, there may be point cloud data or mesh data of various densities, and there may be sparse point cloud data or mesh data and dense point cloud data or mesh data.

[0096] Each processing unit will be described in detail below. The sensor information is acquired by various methods, such as a distance sensor such as a LIDAR or a range finder, a stereo camera, or a combination of multiple monocular cameras. The three-dimensional data generation unit 118 generates three-dimensional data based on the sensor information acquired by the sensor information acquisition unit 117. The three-dimensional data generation unit 118 generates position information as three-dimensional data, and adds attribute information for the position information to the position information.

[0097] The three-dimensional data generation unit 118 may process the three-dimensional data when generating position information or adding attribute information. For example, the three-dimensional data generation unit 118 may reduce the amount of data by deleting point clouds with overlapping positions. The three-dimensional data generation unit 118 may also transform (position shift, rotation, normalization, etc.) the position information. The three-dimensional data generation unit 118 may also generate mesh data from the point cloud data. The three-dimensional data generation unit 118 may also render the attribute information.

[0098] In FIG. 1, the three-dimensional data generation system 111 is included in the three-dimensional data encoding system 101, but it may be provided independently outside the three-dimensional data encoding system 101.

[0099] The encoding unit 113 generates encoded data by encoding the three-dimensional data. There are the following encoding methods. The first is an encoding method using position information, which will be referred to hereinafter as the first encoding method. The second is an encoding method using a video codec, which will be referred to hereinafter as the second encoding method.

[0100] The decoding unit 124 decodes the encoded data to decode the point cloud data. The multiplexing unit 114 multiplexes the encoded data using an existing multiplexing method to generate multiplexed data. The generated multiplexed data is transmitted or stored. In addition to the PCC encoded data, the multiplexing unit 114 multiplexes other media such as video, audio, subtitles, applications, and files, or reference time information. The multiplexing unit 114 may also multiplex sensor information or attribute information related to the three-dimensional data.

[0101] Multiplexing methods or file formats include ISOBMFF, and ISOBMFF-based transmission methods such as MPEG-DASH, MMT, MPEG-2 TS Systems, and RTP.

[0102] The demultiplexer 123 extracts the coded data, other media, time information, and the like from the multiplexed data.

[0103] The input / output unit 115 transmits the multiplexed data using a method suited to the transmission medium or storage medium, such as broadcasting or communication. The input / output unit 115 may communicate with other devices via the Internet, or may communicate with a storage unit such as a cloud server.

[0104] The communication protocol used may be http, ftp, TCP, UDP, etc. A PULL type communication method or a PUSH type communication method may be used.

[0105] Either wired transmission or wireless transmission may be used. For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), coaxial cable, etc. are used. For wireless transmission, wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), millimeter waves, etc. are used.

[0106] As a broadcasting system, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 is used.

[0107] [First Encoding Method] Hereinafter, a method for encoding and decoding a three-dimensional point cloud or a three-dimensional mesh will be described. When the device, process, or syntax in this disclosure relates to encoding or decoding processing of point cloud data, it can also be applied to encoding or decoding of vertices in a three-dimensional mesh. Furthermore, the disclosure regarding encoding or decoding processing of vertices in a three-dimensional mesh can also be applied to encoding or decoding of point cloud data. Furthermore, the disclosure regarding encoding or decoding processing of attribute information of point cloud data may also be applied to encoding or decoding processing of face information, or attribute information for faces or vertices in a three-dimensional mesh. Furthermore, the processing may be shared between point cloud encoding and mesh encoding. This sharing may potentially reduce the scale of the circuit or software.

[0108] Fig. 7 is a diagram showing the configuration of a first encoding unit 130, which is an example of the encoding unit 113 that performs encoding using the first encoding method. Fig. 8 is a block diagram of the first encoding unit 130. The first encoding unit 130 generates encoded data (encoded stream) by encoding point cloud data using the first encoding method. This first encoding unit 130 includes a position information encoding unit 131, an attribute information encoding unit 132, an additional information encoding unit 133, and a multiplexing unit 134.

[0109] The first encoding unit 130 is characterized in that it performs encoding with consideration of a three-dimensional structure. The first encoding unit 130 is also characterized in that the attribute information encoding unit 132 performs encoding using information obtained from the position information encoding unit 131. The first encoding method is also called G-PCC (Geometry-based PCC).

[0110] The point cloud data is PCC point cloud data such as a PLY file or PCC point cloud data generated from sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData). The position information is input to a position information encoding unit 131, the attribute information is input to an attribute information encoding unit 132, and the additional information is input to an additional information encoding unit 133.

[0111] When mesh data is encoded, the position information of the vertex information is input to a position information encoding unit 131 , and the face information or attribute information for the faces and vertices is input to an attribute information encoding unit 132 .

[0112] The position information encoding unit 131 encodes the position information to generate encoded position information (Compressed Geometry), which is encoded data. For example, the position information encoding unit 131 encodes the position information using an N-ary tree structure such as an octtree. Specifically, in an octtree, the target space is divided into eight nodes (subspaces), and 8-bit information (occupancy code) indicating whether or not a point cloud is included in each node is generated. Furthermore, the node including the point cloud is further divided into eight nodes, and 8-bit information indicating whether or not a point cloud is included in each of the eight nodes is generated. This process is repeated until the number of point clouds included in a predetermined layer or node falls below a threshold.

[0113] The attribute information encoding unit 132 generates encoded attribute information (Compressed Attribute) as encoded data by encoding using the configuration information generated by the position information encoding unit 131. For example, the attribute information encoding unit 132 determines a reference point (reference node) to be referenced when encoding a target point (target node) to be processed, based on the octree structure generated by the position information encoding unit 131. For example, the attribute information encoding unit 132 references a peripheral node or adjacent node whose parent node in the octree is the same as that of the target node. Note that the method of determining the reference relationship is not limited to this.

[0114] Furthermore, the encoding process of the attribute information may include at least one of a quantization process, a prediction process, and an arithmetic coding process. In this case, the reference means using a reference node to calculate a predicted value of the attribute information, or using the state of the reference node (e.g., occupancy information indicating whether the reference node includes a point group) to determine an encoding parameter. For example, the encoding parameter may be a quantization parameter in a quantization process, or a context in arithmetic coding.

[0115] The additional information encoding unit 133 generates encoded additional information (Compressed MetaData) that is encoded data by encoding compressible data from the additional information.

[0116] The multiplexing unit 134 multiplexes the encoding position information, the encoding attribute information, the encoding additional information, and other additional information to generate an encoded stream (Compressed Stream) that is encoded data. The generated encoded stream is output to a processing unit of a system layer (not shown).

[0117] Next, a first decoding unit 140, which is an example of the decoding unit 124 that performs decoding using the first encoding method, will be described. FIG. 9 is a diagram showing the configuration of the first decoding unit 140. FIG. 10 is a block diagram of the first decoding unit 140. The first decoding unit 140 generates point cloud data by decoding, using the first encoding method, coded data (coded stream). The first decoding unit 140 includes a demultiplexing unit 141, a position information decoding unit 142, an attribute information decoding unit 143, and an additional information decoding unit 144.

[0118] A coded stream (compressed stream) that is coded data is input to the first decoding unit 140 from a processing unit of a system layer (not shown).

[0119] The demultiplexer 141 separates the encoded position information (Compressed Geometry), the encoded attribute information (Compressed Attribute), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0120] The position information decoding unit 142 generates position information by decoding the encoded position information. For example, the position information decoding unit 142 restores position information of a point group represented by three-dimensional coordinates from the encoded position information represented by an N-ary tree structure such as an octree.

[0121] The attribute information decoding unit 143 decodes the encoded attribute information based on the configuration information generated by the position information decoding unit 142. For example, the attribute information decoding unit 143 determines a reference point (reference node) to be referenced in decoding the target point (target node) to be processed based on the octree structure obtained by the position information decoding unit 142. For example, the attribute information decoding unit 143 references a peripheral node or adjacent node whose parent node in the octree is the same as that of the target node. Note that the method of determining the reference relationship is not limited to this.

[0122] Furthermore, the attribute information decoding process may include at least one of an inverse quantization process, a prediction process, and an arithmetic decoding process. In this case, the reference means using a reference node to calculate a predicted value of the attribute information, or using the state of the reference node (e.g., occupancy information indicating whether the reference node includes a point group) to determine a decoding parameter. For example, the decoding parameter is a quantization parameter in an inverse quantization process, or a context in an arithmetic decoding process.

[0123] The additional information decoding unit 144 generates additional information by decoding the encoded additional information. The first decoding unit 140 uses the additional information necessary for decoding the position information and attribute information during decoding, and outputs the additional information necessary for the application to the outside.

[0124] [Second Encoding Method] Next, a second encoding unit 150, which is an example of the encoding unit 113 that performs encoding using the second encoding method, will be described. Fig. 11 is a diagram showing the configuration of the second encoding unit 150. Fig. 12 is a block diagram of the second encoding unit 150.

[0125] The second encoding unit 150 generates encoded data (encoded stream) by encoding the point cloud data using a second encoding method. The second encoding unit 150 includes an additional information generation unit 151, a position image generation unit 152, an attribute image generation unit 153, a video encoding unit 154, an additional information encoding unit 155, and a multiplexing unit 156.

[0126] The second encoding unit 150 generates a position image and an attribute image by projecting a three-dimensional structure onto a two-dimensional image, and encodes the generated position image and attribute image using an existing video encoding method. The second encoding method is also called VPCC (Video-based PCC).

[0127] The point cloud data is PCC point cloud data such as a PLY file or PCC point cloud data generated from sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData).

[0128] The additional information generating unit 151 generates map information of a plurality of two-dimensional images by projecting a three-dimensional structure onto the two-dimensional images.

[0129] The position image generating unit 152 generates a position image (geometry image) based on the position information and the map information generated by the additional information generating unit 151. This position image is, for example, a distance image in which distance (depth) is indicated as pixel values. Note that this distance image may be an image in which multiple point clouds are viewed from a single viewpoint (an image in which multiple point clouds are projected onto a single two-dimensional plane), or may be multiple images in which multiple point clouds are viewed from multiple viewpoints, or may be a single image in which these multiple images are integrated.

[0130] The attribute image generation unit 153 generates an attribute image based on the attribute information and the map information generated by the additional information generation unit 151. This attribute image is, for example, an image in which attribute information (e.g., color (RGB)) is represented as pixel values. Note that this image may be an image in which multiple point clouds are viewed from a single viewpoint (an image in which multiple point clouds are projected onto a single two-dimensional plane), or multiple images in which multiple point clouds are viewed from multiple viewpoints, or a single image in which these multiple images are integrated.

[0131] When mesh data is encoded, the position information of the vertex information is input to the position image generating unit 152 , and the face information or attribute information for the faces and vertices is input to the attribute image generating unit 153 .

[0132] The video encoding unit 154 encodes the position image and the attribute image using a video encoding method to generate an encoded position image (Compressed Geometry Image) and an encoded attribute image (Compressed Attribute Image), which are encoded data. Note that any known encoding method may be used as the video encoding method. For example, the video encoding method is AVC, HEVC, or the like.

[0133] The additional information encoding unit 155 generates encoded additional information (Compressed MetaData) by encoding the additional information, map information, and the like included in the point cloud data.

[0134] The multiplexing unit 156 multiplexes the encoding position image, the encoding attribute image, the encoding additional information, and other additional information to generate an encoded stream (Compressed Stream) that is encoded data. The generated encoded stream is output to a processing unit of a system layer (not shown).

[0135] Next, a second decoding unit 160, which is an example of the decoding unit 124 that performs decoding using the second encoding method, will be described. FIG. 13 is a diagram showing the configuration of the second decoding unit 160. FIG. 14 is a block diagram of the second decoding unit 160. The second decoding unit 160 generates point cloud data by decoding, using the second encoding method, coded data (coded stream). The second decoding unit 160 includes a demultiplexing unit 161, a video decoding unit 162, an additional information decoding unit 163, a position information generation unit 164, and an attribute information generation unit 165.

[0136] A coded stream (compressed stream) that is coded data is input to the second decoding unit 160 from a processing unit of a system layer (not shown).

[0137] The demultiplexing unit 161 separates the encoded position image (Compressed Geometry Image), the encoded attribute image (Compressed Attribute Image), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0138] The video decoding unit 162 generates a position image and an attribute image by decoding the encoded position image and the encoded attribute image using a video encoding method. Note that any known encoding method may be used as the video encoding method. For example, the video encoding method may be AVC or HEVC.

[0139] The additional information decoding unit 163 decodes the encoded additional information to generate additional information including map information and the like.

[0140] The position information generating unit 164 generates position information using the position image and map information. The attribute information generating unit 165 generates attribute information using the attribute image and map information.

[0141] The second decoding unit 160 uses the additional information necessary for decoding during decoding, and outputs the additional information necessary for the application to the outside.

[0142] [Position Information Encoding in First Encoding Method] Fig. 15 is a block diagram showing an example configuration of the position information encoding unit 131. The position information encoding unit 131 includes an octree encoding unit 171 and a predictive tree encoding unit 172. The octree encoding unit 171 generates encoded position information and metadata by encoding the position information using an encoding method that uses an octree (octree encoding). The predictive tree encoding unit 172 generates encoded position information and metadata by encoding the position information using an encoding method that uses a predictive tree (predictive tree encoding).

[0143] The position information encoding unit 131 encodes the position information using either or both of octree encoding and predictive tree encoding. The position information encoding unit 131 may switch between these two encoding methods, may use an encoding method other than these two methods, or may include the position information in the bitstream as raw data without encoding it. Information indicating the encoding method of the encoded data is stored in metadata and notified to the decoding device.

[0144] 16 is a block diagram showing an example configuration of the position information decoding unit 142. The position information decoding unit 142 includes an octree decoding unit 173 and a predictive tree decoding unit 174. The octree decoding unit 173 generates position information by decoding the encoded position information using a decoding method that uses an octree (octree decoding). The predictive tree decoding unit 174 generates position information by decoding the encoded position information using a decoding method that uses a predictive tree (predictive tree decoding). Furthermore, the position information decoding unit 142 performs decoding using the encoding method notified by the metadata.

[0145] Next, an example of the configuration of the position information encoding unit will be described. Fig. 17 is a block diagram of an octree encoding unit 171 according to this embodiment. The octree encoding unit 171 includes an octree generation unit 181, a geometric information calculation unit 182, a coding table selection unit 183, and an entropy encoding unit 184.

[0146] The octree generation unit 181 generates, for example, an octree from the input position information and generates an occupancy code for each node in the octree. The geometric information calculation unit 182 acquires information indicating whether an adjacent node of the target node is an occupied node. For example, the geometric information calculation unit 182 calculates the occupancy information of the adjacent node (information indicating whether the adjacent node is an occupied node) from the occupancy code of the parent node to which the target node belongs. Alternatively, the geometric information calculation unit 182 may store encoded nodes in a list and search for adjacent nodes from the list. Note that the geometric information calculation unit 182 may switch adjacent nodes depending on the position of the target node within the parent node.

[0147] The coding table selection unit 183 selects a coding table to be used for entropy coding of the target node using the occupancy information of the adjacent nodes calculated by the geometric information calculation unit 182. For example, the coding table selection unit 183 may generate a bit string using the occupancy information of the adjacent nodes, and select a coding table for an index number generated from the bit string.

[0148] The entropy coding unit 184 generates the coding position information and the metadata by entropy coding the occupancy code of the target node using the coding table of the selected index number. The entropy coding unit 184 may add information indicating the selected coding table to the coding position information.

[0149] The octree representation and the scanning order of position information will be described below. Position information (position data) is converted into an octree structure (octreeing) and then encoded. The octree structure is composed of nodes and leaves. Each node has eight nodes or leaves, and each leaf has voxel (VXL) information. FIG. 18 is a diagram showing an example of the structure of position information containing multiple voxels. FIG. 19 is a diagram showing an example of the position information shown in FIG. 18 converted into an octree structure. Here, among the leaves shown in FIG. 19, leaves 1, 2, and 3 represent voxels VXL1, VXL2, and VXL3 shown in FIG. 18, respectively, and represent a VXL containing a point cloud (hereinafter referred to as a valid VXL).

[0150] Specifically, node 1 corresponds to the entire space containing the position information in Figure 18. The entire space corresponding to node 1 is divided into eight nodes, and of the eight nodes, a node containing a valid VXL is further divided into eight nodes or leaves, and this process is repeated for each level of the tree structure. Here, each node corresponds to a subspace, and has information (occupancy code) as node information indicating the position of the next node or leaf after division. In addition, the block at the bottom level is set as a leaf, and the number of point clouds contained in the leaf, etc., is held as leaf information.

[0151] Next, an example of the configuration of the position information decoding unit will be described. Fig. 20 is a block diagram of the octree decoding unit 173 according to this embodiment. The octree decoding unit 173 includes an octree generation unit 191, a geometric information calculation unit 192, a coding table selection unit 193, and an entropy decoding unit 194.

[0152] The octree generator 191 generates an octree of a certain space (node) using header information or metadata of a bitstream. For example, the octree generator 191 generates a large space (root node) using the sizes of the space in the x-axis, y-axis, and z-axis directions added to the header information, and then generates an octree by dividing the space into two in the x-axis, y-axis, and z-axis directions to generate eight small spaces A (nodes A0 to A7). Nodes A0 to A7 are set in order as target nodes.

[0153] The geometric information calculation unit 192 acquires occupancy information indicating whether an adjacent node of a target node is an occupied node. For example, the geometric information calculation unit 192 calculates the occupancy information of the adjacent node from the occupancy code of the parent node to which the target node belongs. Alternatively, the geometric information calculation unit 192 may store decoded nodes in a list and search for adjacent nodes from the list. Note that the geometric information calculation unit 192 may switch adjacent nodes depending on the position of the target node within the parent node.

[0154] The coding table selection unit 193 selects a coding table (decoding table) to be used for entropy decoding of the target node using the occupancy information of the adjacent nodes calculated by the geometric information calculation unit 192. For example, the coding table selection unit 193 may generate a bit string using the occupancy information of the adjacent nodes, and select a coding table for an index number generated from the bit string.

[0155] The entropy decoding unit 194 generates position information by entropy decoding the occupancy code of the target node using the selected coding table. Note that the entropy decoding unit 194 may obtain information on the selected coding table by decoding it from the bitstream, and entropy decode the occupancy code of the target node using the coding table indicated by the obtained information.

[0156] [Attribute Information Encoding in First Encoding Method] The configurations of the attribute information encoding unit and the attribute information decoding unit will be described below. Fig. 21 is a block diagram showing an example configuration of the attribute information encoding unit 132. The attribute information encoding unit may include multiple encoding units that execute different encoding methods. For example, the attribute information encoding unit may switch between the following two methods depending on the use case:

[0157] The attribute information encoding unit 132 includes an LoD attribute information encoding unit 201 and a conversion attribute information encoding unit 202. The LoD attribute information encoding unit 201 classifies each 3D point into multiple layers using the position information of the 3D point, predicts the attribute information of the 3D point belonging to each layer, and encodes the prediction residual. Here, each classified layer is called LoD (Level of Detail).

[0158] The transformed attribute information encoding unit 202 encodes the attribute information using RAHT (Region Adaptive Hierarchical Transform). Specifically, the transformed attribute information encoding unit 202 applies RAHT or Haar transform to each piece of attribute information based on the position information of the three-dimensional points to generate high-frequency components and low-frequency components for each layer, and encodes these values ​​using quantization, entropy coding, or the like.

[0159] 22 is a block diagram showing an example configuration of the attribute information decoding unit 143. The attribute information decoding unit may include multiple decoding units that execute different decoding methods. For example, the attribute information decoding unit may switch between the following two methods for decoding based on information included in the header or metadata.

[0160] The attribute information decoding unit 143 includes an LoD attribute information decoding unit 203 and a converted attribute information decoding unit 204. The LoD attribute information decoding unit 203 classifies each 3D point into multiple layers using the position information of the 3D points, and decodes the attribute values ​​while predicting the attribute information of the 3D points belonging to each layer.

[0161] The transformed attribute information decoding unit 204 decodes the attribute information using RAHT (Region Adaptive Hierarchical Transform). Specifically, the transformed attribute information decoding unit 204 decodes the attribute values ​​by applying an inverse RAHT or an inverse Haar transform to the high-frequency components and low-frequency components of each attribute value based on the position information of the three-dimensional points.

[0162] 23 is a block diagram of a transformed attribute information coding unit 202, which is an example of the transformed attribute information coding unit 202. The transformed attribute information coding unit 202 includes a sorting unit 211, a Haar transform unit 212, a quantization unit 213, an inverse quantization unit 214, an inverse Haar transform unit 215, a memory 216, and an arithmetic coding unit 217.

[0163] The sorting unit 211 generates a Morton code using the position information of the 3D points and sorts the multiple 3D points in Morton code order. The Haar transform unit 212 generates coding coefficients by applying a Haar transform to the attribute information. The quantization unit 213 quantizes the coding coefficients of the attribute information.

[0164] The inverse quantization unit 214 inversely quantizes the quantized coding coefficients. The inverse Haar transform unit 215 applies inverse Haar transform to the coding coefficients. The memory 216 stores values ​​of attribute information of multiple decoded 3D points. For example, the attribute information of the decoded 3D points stored in the memory 216 may be used for predicting uncoded 3D points.

[0165] The arithmetic coding unit 217 calculates ZeroCnt from the quantized coding coefficients and arithmetically codes the ZeroCnt. The arithmetic coding unit 217 also arithmetically codes the quantized non-zero coding coefficients. The arithmetic coding unit 217 may binarize the coding coefficients before arithmetic coding. The arithmetic coding unit 217 may also generate and code various header information.

[0166] 24 is a block diagram of the transformed attribute information decoding unit 204, which is an example of the transformed attribute information decoding unit 204. The transformed attribute information decoding unit 204 includes an arithmetic decoding unit 221, an inverse quantization unit 222, an inverse Haar transform unit 223, and a memory 224.

[0167] The arithmetic decoding unit 221 arithmetically decodes the ZeroCnt and the coding coefficients included in the bitstream. Note that the arithmetic decoding unit 221 may also decode various types of header information.

[0168] The inverse quantization unit 222 inverse quantizes the arithmetically decoded coding coefficients. The inverse Haar transform unit 223 applies inverse Haar transform to the inversely quantized coding coefficients. The memory 224 stores values ​​of attribute information of multiple decoded 3D points. For example, the attribute information of the decoded 3D points stored in the memory 224 may be used to predict undecoded 3D points.

[0169] [Slice Division] The encoding device may divide the three-dimensional data into one or more three-dimensional data and encode the divided three-dimensional point cloud. The divided three-dimensional point cloud is called a slice. A slice is a set of points each having position information (geometry) and attribute information (attribute).

[0170] 25 is a block diagram showing an example of the configuration of a three-dimensional data encoding device in this case. The three-dimensional data encoding device includes a data dividing unit 231 and an encoding unit 232.

[0171] The data dividing unit 231 divides the three-dimensional data to generate a plurality of divided three-dimensional data. Each divided three-dimensional data corresponds to a slice. The encoding unit 232 encodes the position information and attribute information of each of the plurality of divided three-dimensional data (slices) to generate encoded data.

[0172] 26 is a block diagram showing an example configuration of a three-dimensional data decoding device in this case. The three-dimensional data decoding device includes a decoding unit 233 and a data combining unit 234. The decoding unit 233 generates a plurality of divided three-dimensional data by decoding the encoded data. Each divided three-dimensional data includes position information and attribute information of a slice. The data combining unit 234 restores the three-dimensional data by combining the plurality of divided three-dimensional data.

[0173] In encoding, slices may or may not have dependencies. If there are no dependencies, slices can be encoded or decoded independently. Therefore, parallel processing of multiple slices can reduce processing time. In addition, partial decoding, which decodes only some slices, can reduce the amount of processing.

[0174] If there is a dependency, an identifier indicating the dependency is stored in the bitstream, and encoding or decoding is performed in order starting from the dependent data (referenced data).

[0175] The number of divisions or the division method may be any method. The three-dimensional data encoding device may determine the shape of an object and divide the point cloud for each object, or may perform division based on the number of points included in a slice. Furthermore, the three-dimensional data encoding device may use map information or location information to perform division based on whether the point cloud is included in the three-dimensional space (tile information).

[0176] Fig. 27 is a diagram showing the relationship between tiles and slices. As shown in Fig. 27, tiles correspond to three-dimensional space, and slices correspond to divided three-dimensional point clouds. Note that multiple tiles may overlap. This enables parallel processing in adaptive encoding or decoding according to content or object, improving the flexibility of the point cloud encoding system or point cloud decoding system.

[0177] [Structure of Encoded Data] The position information and attribute information for each slice are encoded. At least a part of the resulting encoded data is stored in the payload of a data unit. A header is also added to the payload.

[0178]

[0043] When multiple pieces of attribute information exist for each point, each of the multiple pieces of attribute information is stored in a data unit. FIG. 28 is a diagram showing an example of the configuration of a bit stream (encoded data). FIG. 28 shows encoded data of two frames of point clouds having two types of attribute information. The bit stream has a geometry (position information) data unit (Geometry Data Unit: Geom) and two attribute (attribute information) data units (Attribute Data Unit: Attr(0), Attr(1)) for each slice. For example, when two pieces of attribute information, color and reflectance, are present for each point, the encoded data for color is stored in Attr(0), and the encoded data for reflectance is stored in Attr(1). The header of the geometry data unit stores the slice identifier (slice_id). The header of the attribute data unit indicates the slice_id of the corresponding (referring to) geometry data unit. The data unit is sometimes called a slice, and the data unit header is sometimes called a slice header.

[0179] Furthermore, metadata related to encoding of position information is stored in a geometry (position information) parameter set (GPS). Metadata related to encoding of attribute information is stored in an attribute parameter set (APS). Metadata common to multiple PCC frames (PCC sequences) is stored in a sequence parameter set (SPS).

[0180] Each data unit and parameter set is converted into a NAL (Network Access Layer) unit or a TLV (Type Length Value) unit format, and a data sequence (stream) of the units is output.

[0181] 29 is a block diagram showing an example configuration of a three-dimensional data encoding device that outputs a stream of TLV units. The three-dimensional data encoding device includes an encoding unit 241 and a TLV storage unit 242. The encoding unit 241 generates encoded data by encoding point cloud data. The TLV storage unit 242 generates a TLV stream (a single stream of TLV units) by storing (encapsulating) the encoded data in multiple TLV units.

[0182] A unit includes a Type that indicates the data type, a Length that indicates length information, and a Value that stores the data indicated by the Type. Note that a unit may have a different format, such as not including length information.

[0183] Figure 28 shows a sequence of units that store data units and parameter sets. The arrows in Figure 28 indicate dependencies related to the decoding of encoded data. The source of the arrow depends on the data at the end of the arrow, and the header of the original data indicates an identifier for the data at the end of the arrow (reference destination). The three-dimensional data decoding device decodes the data at the end of the arrow and uses the decoded data to decode the data at the source of the arrow. For example, a geometry data unit (Geom) indicates the GPS ID and SPS ID corresponding to the geometry data unit. An attribute data unit (Attr) indicates the APS ID corresponding to the attribute data unit.

[0184] One GPS and one APS may be provided for each frame. Alternatively, if the encoding method is changed for each slice, one GPS and one APS may be provided for each slice. Furthermore, the GPS and the APS may be shared by multiple frames (sequences).

[0185] In addition, the APS may be shared by multiple pieces of attribute information. The parameter set of the reference destination is sent before the parameter set of the reference source.

[0186] When inter-frame prediction is used, for example, if there is a dependency between data units of frames, a GOF (Group of Frame) including multiple frames may be configured. A GOF is a random access unit, and the first slice of a GOF has no dependency and is the slice from which decoding starts. The stream may have a delimiter indicating a frame boundary or a GOF boundary, or a TLV unit indicating the boundary.

[0187] Next, the structure of the coded data and the method of storing the coded data in the NAL unit will be described.

[0188] For example, a data format is defined for each type of encoded data. Fig. 30 shows examples of encoded data and NAL units.

[0189] For example, as shown in Figure 30, the encoded data includes a header and a payload. The encoded data may include length information indicating the length (amount of data) of the encoded data, header, or payload. The encoded data may be in a TLV format including the length information. The encoded data may not include a header.

[0190] The header includes, for example, identification information for identifying the data, such as the data type or frame number.

[0191] The header includes, for example, identification information indicating a reference relationship. This identification information is stored in the header when, for example, there is a dependency relationship between data, and is information for referencing the reference destination from the reference source. For example, the header of the reference destination includes identification information for identifying the data. The header of the reference source includes identification information indicating the reference destination.

[0192] If the reference destination or the reference source can be identified or derived from other information, the identification information for specifying the data or the identification information indicating the reference relationship may be omitted.

[0193] The three-dimensional data encoding device stores encoded data in the payload of a NAL unit. The NAL unit header includes pcc_nal_unit_type, which is identification information of the encoded data. Figure 31 is a diagram showing an example of the semantics of pcc_nal_unit_type.

[0194] As shown in FIG. 31, when pcc_codec_type is codec 1 (Codec1: first encoding method), the values ​​0 to 10 of pcc_nal_unit_type correspond to the encoded position data (Geometry), encoded attribute X data (AttributeX), encoded attribute Y data (AttributeY), position PS (Geom.PS), attribute XPS (AttrX.PS), attribute YPS (AttrY.PS), position SPS (Geometry Sequence PS), attribute XSPS (AttributeX Sequence PS), attribute YSPS (AttributeY Sequence PS), AU header (AU Header), and GOF header (GOF Values ​​11 and above are assigned as spares for Codec 1.

[0195] A modified example of this embodiment will be described below. PS has levels, such as a frame-level PS, a sequence-level PS, and a PCC sequence-level PS, and if the PCC sequence level is the higher level and the frame level is the lower level, the following method may be used to store parameters.

[0196] The default PS value is indicated in the higher PS. Also, if the value of the lower PS differs from the value of the higher PS, the PS value is indicated in the lower PS. Alternatively, the PS value is not written in the higher PS, but written in the lower PS. Alternatively, information on whether the PS value is to be written in the lower PS, the higher PS, or both is written in either or both of the lower PS and the higher PS. Alternatively, the lower PS may be merged with the higher PS. Alternatively, if the lower PS and the higher PS overlap, the three-dimensional data encoding device may omit sending one of them.

[0197] The three-dimensional data encoding device may divide data into slices or tiles and transmit the divided data. The divided data includes information for identifying the divided data, and parameters used to decode the divided data are included in the parameter set. In this case, an identifier indicating that the data stores data or parameters related to tiles or slices is defined in pcc_nal_unit_type.

[0198] [Gaussian Data] Gaussian data (Gaussian Splatting 3D data) will be described below. First, the configuration of a system that encodes or decodes three-dimensional data generated using the Gaussian Splatting method will be described.

[0199] Fig. 32 is a diagram showing a part of the processing in the three-dimensional data generation system shown in Fig. 1. A sensor information input unit 301 and a three-dimensional data generation unit 302 shown in Fig. 32 correspond to, for example, the sensor information acquisition unit 117 and the three-dimensional data generation unit 118 shown in Fig. 1.

[0200] The sensor information input unit 301 acquires point cloud data (three-dimensional point cloud data) and outputs the acquired point cloud data to the three-dimensional data generation unit 302. The three-dimensional data generation unit 302 generates Gaussian data (Gaussian splatting data) from the point cloud data and outputs the generated Gaussian data.

[0201] Specifically, the three-dimensional data generation unit 302 first generates a grid according to the density of the input point cloud data. Each point of the point cloud data is projected onto a nearby grid. At this time, attribute information associated with the point, such as color, reflectance, or normal, is also simultaneously associated with the grid. The three-dimensional data generation unit 302 derives parameters such as the mean value and variance of the Gaussian function from the information projected onto the grid.

[0202] Note that the method for generating Gaussian data is not limited to the above. For example, Gaussian data may be generated by machine learning using point cloud data generated from multiple two-dimensional images using structure from motion (SfM) or the like.

[0203] Gaussian data is a collection of multiple 3D Gaussians (ellipsoids). Each 3D Gaussian includes data such as three-dimensional point coordinates, a 3x3 covariance matrix, color, and transparency. The above 3D Gaussian data can also be converted into components described in the PLY format and expressed.

[0204] Fig. 33 is a diagram showing the components of Gaussian data described in the PLY format. The Gaussian data shown in Fig. 33 includes position, rotation, scale, SH coefficient, and transparency. Note that transparency is also called transparency, transparency, or opacity.

[0205] Position is the three-dimensional coordinate of a three-dimensional point. Scale and Orientation are parameters of a three-dimensional covariance matrix. SH Coefficient is the coefficient of a spherical harmonic function when color information is expressed as a spherical harmonic function.

[0206] That is, the Gaussian data includes, for each 3D Gaussian, three-dimensional coordinates (position) that serve as a reference, and information associated with the three-dimensional coordinates (rotation, scale, SH coefficient, transmittance).

[0207] The configuration of the Gaussian data is not limited to the above. For example, the Gaussian data does not need to include a covariance matrix. For example, the Gaussian data includes a position (Position) indicating the three-dimensional coordinates of a three-dimensional point, a scale (Scale) indicating the scale along each axis, a color (Color) indicating, for example, RGB, and a transparency (Transparency).

[0208] Furthermore, the Gaussian data may include a plurality of cubes (voxels) that divide a 3D space instead of a plurality of ellipsoids. Each voxel has attribute information. For example, each voxel may include a position (Position) indicating the three-dimensional coordinates of the center point of the voxel, attribute information such as color, reflectance, or material, and a scale (Scale) indicating the size of the voxel.

[0209] Furthermore, the position (Position) may be represented by two-dimensional point coordinates (x, y) instead of three-dimensional coordinates. In this case, the scale (Scale) may indicate the size or shape of the ellipsoid on a two-dimensional plane. Specifically, the Gaussian data may include a position (Position) indicating a position on a two-dimensional plane, a scale vector (Scale Vector) indicating the scale (Su, Sv) along each axis, and rotation information (Rotation) indicating the orientation of the ellipsoid.

[0210] The format for writing Gaussian data is not limited to the PLY format, but may be, for example, the SPZ format.

[0211] [Gaussian Data Rendering Process] Next, the Gaussian data rendering process will be described. FIG. 34 is a diagram showing the Gaussian data rendering process. As shown in FIG. 34, the rendering unit 303 generates a 3D model (three-dimensional model) by rendering the Gaussian data. This makes it possible to present the generated 3D model in an application such as a 3D display device. The 3D display device is, for example, a device that displays a stereoscopic image in space like a hologram, or a naked-eye 3D display device.

[0212] 35 is a diagram showing an example of the number of SH coefficients of a spherical harmonic function. The number of elements (the number of SH coefficients) of a spherical harmonic function is determined according to the level indicating the resolution. For example, a higher level (a larger numerical value of the level) indicates a higher resolution. For example, the higher the level, the more information of high frequency components is added.

[0213] For example, when expressing level 2, the spherical harmonic function has a total of nine elements (SH coefficients) from levels 0 to 2. For example, when expressing level 3, the spherical harmonic function has a total of 16 elements (SH coefficients) from levels 0 to 3.

[0214] For example, the example shown in Figure 33 shows a spherical harmonic function for a level 3 color, where each of the three color elements (R, G, B) is expressed by 16 SH coefficients. Therefore, the spherical harmonic function has 16 x 3 = 48 elements. Note that levels 4 and above can also be defined.

[0215] By expressing color using spherical harmonic functions, it is possible to express three-dimensional color information. Fig. 36 is a diagram showing an example of input and output of a spherical harmonic function. For example, as shown in Fig. 36, by inputting viewpoint information into a color spherical harmonic function 304, color information (R, G, B) seen from that viewpoint is output.

[0216] 37 is a diagram showing the rendering process. As shown in FIG. 37, the rendering unit 305 generates a 3D model or a 2D image viewed from a specified viewpoint based on Gaussian data and the specified viewpoint. The generated 3D model can be presented in an application such as a VR display. The generated 2D image can also be presented in an application such as a 2D image display device.

[0217] Note that, in addition to color, other attribute information (such as reflectance or infrared information) may be expressed using spherical harmonic functions. In this case, the Gaussian data includes SH coefficients of the spherical harmonic functions for each attribute information. Furthermore, the level may be changed depending on the attribute information or resolution.

[0218] [Encoding and Decoding of Gaussian Data] Next, encoding and decoding of Gaussian data will be described. Fig. 38 is a block diagram of an encoding device (encoding system) according to this embodiment. This encoding device encodes and multiplexes Gaussian data. Fig. 39 is a block diagram of a decoding device (decoding system) according to this embodiment. This decoding device decodes the Gaussian data and presents a 3D model or a 2D image in an application. The encoding device and decoding device shown in Figs. 38 and 39 are included in, for example, the three-dimensional data generation system shown in Fig. 1.

[0219] 38 includes an encoding unit 311 and a multiplexing unit 312. The encoding unit 311 generates encoded data (bit stream) by encoding input Gaussian data using a predetermined encoding method, and outputs the generated encoded data to the multiplexing unit 312.

[0220] The multiplexing unit 312 multiplexes the input coded data using a predetermined multiplexing method to generate multiplexed data, and outputs the generated multiplexed data. This multiplexed data is stored or transmitted.

[0221] 39 includes a demultiplexing unit 321, a decoding unit 322, and an application unit 323. The demultiplexing unit 321 demultiplexes input multiplexed data using a predetermined multiplexing method to generate coded data, and outputs the generated coded data to the decoding unit 322.

[0222] The decoding unit 322 generates Gaussian data by decoding the input coded data using a predetermined coding method (decoding method), and outputs the generated Gaussian data to the application unit 323 .

[0223] The application unit 323 includes an input interface unit 324, a rendering unit 325, and a presentation unit 326. The input interface unit 324 acquires a user operation. For example, the input interface unit 324 generates viewpoint information indicating a viewpoint input by the user based on the user operation.

[0224] The rendering unit 325 generates a three-dimensional model or a two-dimensional image by rendering the input Gaussian data. For example, the rendering unit 325 generates a 3D model or a 2D image viewed from a viewpoint input by the user, which viewpoint information indicates. Note that the rendering unit 325 may also generate a 3D model or a 2D image viewed from a predetermined viewpoint.

[0225] The presentation unit 326 presents (displays) a three-dimensional model or a two-dimensional image generated by the rendering unit 325. Whether to present a 3D model or a 2D image may be determined, for example, by the following method. For example, the rendering unit 325 first generates a three-dimensional model and generates point coordinate information, as in the process shown in FIG. 34 . After viewpoint information is input, the rendering unit 325 generates a 2D image viewed from a specified viewpoint. In this case, the rendering unit 325 may also receive information specifying an area along with the specified viewpoint. In this way, by first generating a 3D model and then generating a 2D image of the specified area, the amount of processing can be reduced compared to generating a 2D image of the entire area. This allows for increased rendering speed. Alternatively, whether to present a 3D display (3D model) or a 2D image may be determined depending on the processing capabilities of the device performing the rendering. For example, in the case of a device with low processing capabilities, the device may present a 3D model without generating a 2D image.

[0226] (First Aspect) [Encoding of Gaussian Data Using G-PCC] Fig. 40 is a block diagram showing an example configuration of an encoding device 330 (three-dimensional data encoding device) according to this embodiment. The encoding device 330 generates a bit stream (encoded data) by encoding first Gaussian data.

[0227] The encoding device 330 includes a first pre-processing unit 331 , a second pre-processing unit 332 , and a G-PCC encoding unit 333 .

[0228] The first Gaussian data includes, for example, a covariance matrix. The first pre-processing unit 331 converts the first Gaussian data into second Gaussian data composed of elements in a ply format. For example, the elements of a ply file include three-dimensional coordinates, scale, rotation, SH coefficients of spherical harmonics, and transmittance, as shown in FIG.

[0229] Note that, although an example will be described here in which first Gaussian data is input to the encoding device 330 and the encoding device 330 converts the first Gaussian data into second Gaussian data, the second Gaussian data may also be input to the encoding device 330. In this case, the encoding device 330 does not need to include the first pre-processing unit 331.

[0230] The second pre-processing unit 332 converts the multiple components contained in the second Gaussian data into a format that can be encoded using an encoding method corresponding to each component, and outputs the converted multiple components to the G-PCC encoding unit 333.

[0231] The G-PCC encoding unit 333 encodes three-dimensional coordinates using a geometry (position information) encoding method (such as an octet tree encoding method, a predictive tree encoding method, or a TriSoup method) in the G-PCC encoding method (Geometry based PCC). The G-PCC encoding unit 333 encodes information associated with three-dimensional coordinates, such as rotation, scale, SH coefficients, and transmittance, using an attribute encoding method in the G-PCC encoding method (such as an LoD-based encoding method (LoD attribute information encoding method) or a Transform-based encoding method (transform attribute information encoding method)).

[0232] The G-PCC encoding unit 333 includes a position information encoding unit 341, a rotation encoding unit 342, a scale encoding unit 343, an SH coefficient encoding unit 344, a transmittance encoding unit 345, a metadata encoding unit 346, and a multiplexing unit 347.

[0233] The position information encoding unit 341 encodes three-dimensional coordinates. The three-dimensional coordinates are reference or representative three-dimensional coordinates for the Gaussian data. For example, the three-dimensional coordinates may be the center coordinates of an ellipse that constitutes the Gaussian data, or may be other coordinates. For example, the three-dimensional coordinates may be the origin coordinates of a grid used to generate the Gaussian data.

[0234] The three-dimensional coordinates are not processed by the first pre-processing unit 331, but are input directly to the second pre-processing unit 332. The second pre-processing unit 332 converts the input three-dimensional coordinates into positive integer information (three-dimensional coordinates). The scale value and offset value used in the conversion are stored in metadata such as an SPS, and are notified to the decoding device.

[0235] The position information encoding unit 341 generates encoded data by encoding the three-dimensional coordinates output from the second pre-processing unit 332 using the G-PCC geometry encoding method. This encoded data is stored in a GDU (Geometry Data Unit) and output. In addition, metadata required for decoding the GDU is stored in a GPS (Geometry Parameter Set) and output.

[0236] Furthermore, when the three-dimensional coordinates of the multiple Gaussian data are sparse (the density is less than a predetermined threshold), the position information encoding unit 341 may use a predictive tree encoding method suitable for encoding sparse three-dimensional points. Furthermore, when the three-dimensional coordinates of the multiple Gaussian data are dense (the density is equal to or greater than a predetermined threshold), the position information encoding unit 341 may use an octree encoding method suitable for encoding dense three-dimensional points.

[0237] The first pre-processing unit 331 converts the covariance matrix included in the first Gaussian data into a rotation value and a scale value, converts color information included in the first Gaussian data into a spherical harmonic function, and outputs SH coefficients in the spherical harmonic function.

[0238] The second pre-processing unit 332 converts the data type of each piece of data included in the second Gaussian data. Specifically, the second pre-processing unit 332 converts each piece of data into positive integer data in order to encode it using the G-PCC encoding method. For example, if the data is float type, the second pre-processing unit 332 converts the data into positive integer data by performing scaling and offset processing on the data. At this time, the scale value and offset value for each piece of attribute information used in the conversion are stored in metadata such as an SPS and notified to the decoding device.

[0239] The second pre-processing unit 332 also performs format conversion for each data item and mapping processing for mapping each data item to an attribute component. For example, if the maximum number of elements (number of dimensions) per attribute component that can be coded using the G-PCC attribute coding method is three, the second pre-processing unit 332 converts the input configuration element into attribute components with three dimensions (three elements) per attribute component. For example, since scale (Scale) has three elements, it is output as a single attribute component. Furthermore, since rotation (Orientation) has four elements, it is converted into two two-dimensional attribute components. The number of elements for the SH coefficients changes for each level. For example, SH coefficients with levels greater than three are converted into multiple attribute components.

[0240] At this time, the second pre-processing unit 332 generates, as metadata, mapping information indicating to which attribute component each element is mapped for each piece of attribute information.

[0241] The rotation encoding unit 342, scale encoding unit 343, SH coefficient encoding unit 344, and transmittance encoding unit 345 generate a plurality of encoded data by encoding the rotation, scale, SH coefficient, and transmittance, respectively, using an attribute encoding method in the G-PCC encoding method. Each of the generated plurality of encoded data is stored in an ADU (Attribute Data Unit) and output. Metadata required for decoding the ADU is stored in an APS (Attribute Parameter Set) and output. In addition, the attribute information is encoded using three-dimensional coordinates.

[0242] The metadata encoding unit 346 stores transformation information and the like including the transformation parameters used in the first pre-processing unit 331 or the second pre-processing unit 332 in metadata such as an SPS or SEI. Note that the metadata encoding unit 346 may encode the transformation information and then store it in the metadata. Furthermore, the metadata may include configuration information of the Gaussian data or information indicating a correspondence relationship of how the Gaussian data is transmitted by G-PCC encoding.

[0243] The multiplexing unit 347 stores the coded data (ADU and GDU) and the parameter set (APS, GPS, and SPS) in a TLV unit and transmits it as a bit stream (coded data). The bit stream may be multiplexed using a predetermined multiplexing method. The bit stream may also be formatted.

[0244] [Decoding Gaussian Data Using G-PCC] Fig. 41 is a block diagram showing an example configuration of a decoding device 350 (three-dimensional data decoding device) according to this embodiment. The decoding device 350 generates first Gaussian data or second Gaussian data by decoding a bit stream (encoded data). For example, the decoding device 350 decodes the bit stream generated by the encoding device 330 shown in Fig. 40.

[0245] The decoding device 350 includes a G-PCC decoding unit 351 , a second post-processing unit 352 , and a first post-processing unit 353 .

[0246] The G-PCC decoding unit 351 generates three-dimensional coordinates, rotation, scale, SH coefficients, and transmittance by decoding the bitstream using the G-PCC encoding method. The G-PCC decoding unit 351 includes a demultiplexing unit 361, a position information decoding unit 362, a rotation decoding unit 363, a scale decoding unit 364, an SH coefficient decoding unit 365, a transmittance decoding unit 366, and a metadata decoding unit 367.

[0247] The demultiplexing unit 361 analyzes a plurality of TLV units included in the input bitstream and generates data units of coded data such as GDU, ADU, SPS, GPS, APS, etc. The position information decoding unit 362, rotation decoding unit 363, scale decoding unit 364, SH coefficient decoding unit 365, transmittance decoding unit 366, and metadata decoding unit 367 decode these data units using the respective coding methods for each attribute component using information in the parameter set.

[0248] The position information decoding unit 362 decodes three-dimensional coordinates from the GDU. The rotation decoding unit 363, scale decoding unit 364, SH coefficient decoding unit 365, and transmittance decoding unit 366 decode attribute information (rotation, scale, SH coefficient, and transmittance) from the ADU. The attribute information is decoded using the three-dimensional coordinates. The metadata decoding unit 367 acquires transformation information and the like from metadata such as SPS or SEI. The metadata decoding unit 367 may decode the transformation information from the metadata. The metadata may also include configuration information of the Gaussian data or information indicating a correspondence relationship between how the Gaussian data is transmitted using G-PCC encoding.

[0249] The second post-processing unit 352 generates second Gaussian data by inversely transforming the decoded data. Here, the inverse transform is a process opposite to the transform process performed by the second pre-processing unit 332, and is performed based on the transform information (transformation parameters) included in the metadata.

[0250] The first post-processing unit 353 generates first Gaussian data by inversely transforming the second Gaussian data. Here, the inverse transform is a process opposite to the transform process performed by the first pre-processing unit 331. The first post-processing unit 353 outputs at least one of the first Gaussian data and the second Gaussian data. Note that the decoding device 350 does not necessarily have to include the first post-processing unit 353.

[0251] [Mapping Process] The second pre-processing unit 332 maps the Gaussian data to the attribute components of the G-PCC. The mapping method and the mapping information generated at that time will be described.

[0252] In the following, an example of mapping rotation, scale, and SH coefficients will be shown, and a description of transmittance will be omitted. Note that similar processing can be applied when other attribute information is added.

[0253] 42 is a diagram showing an example of mapping multiple attribute components included in Gaussian data to attribute components of G-PCC, which is a component of the encoding method. In this example, the maximum number of dimensions (number of subcomponents) that can be encoded in encoding of attribute components in G-PCC is 3.

[0254] In the following description, the attribute component of Gaussian data may be referred to as a Gaussian component, and the attribute component of G-PCC may be referred to as a G-PCC component. Also, the attribute component may be simply referred to as a component.

[0255] For example, a Gaussian component of a scale has subcomponents (B1 to B3) with a dimensionality of 3, and is mapped to one G-PCC component having three-dimensional subcomponents. In this case, the mapping information indicates that the identifiers of the Gaussian component of the scale and the identifiers of the G-PCC component correspond to each other.

[0256] Furthermore, the Gaussian rotation component has subcomponents (A1 to A4) with a dimensionality of four, and cannot be mapped to a single G-PCC component with three-dimensional subcomponents. Therefore, the four rotation subcomponents (A1 to A4) are mapped to two G-PCC components with two-dimensional subcomponents. In this case, the mapping information indicates that the identifier of the Gaussian rotation component and the identifier indicating the subcomponent number included in the attribute component correspond to the identifier of the G-PCC component and the identifier indicating the subcomponent number included in the G-PCC component.

[0257] Since there are 16 SH coefficients per level for each three-dimensional (RGB) element, the SH coefficients have a total of 48 dimensional elements (C1 to C48), which are mapped to 16 G-PCC components with three-dimensional subcomponents.

[0258] 42, "ID" is an attribute component ID, which indicates an identification number for identifying an attribute component of encoded data. "ND" is the number of dimensions, which indicates the number of elements (number of subcomponents) per attribute component.

[0259] Furthermore, the division method and mapping method of Gaussian data are not limited to the above examples, and any combination of mapping method and number of dimensions may be used.

[0260] In the mapping process, the second pre-processing unit 332 generates mapping information indicating the G-PCC components corresponding to the Gaussian components, and stores the mapping information as metadata in the header or SEI.

[0261] The decoding device decodes the mapping information from the bitstream. The second post-processing unit 352 remaps the decoded data for each G-PCC component to Gaussian data based on the mapping information, thereby reconstructing the Gaussian data.

[0262] Fig. 43 is a diagram showing an example of mapping information. In Fig. 43, the mapping information is stored in SEI (Gaussian data attribute mapping information SEI). The mapping information includes the number of Gaussian data components. The number of Gaussian data components indicates the number of attribute information included in the Gaussian data. For example, in the example shown in Fig. 42, rotation, scale, and SH coefficients are present, and the number of Gaussian data components is 3.

[0263] The mapping information also includes, as information for each Gaussian component, a Gaussian data component ID, a Gaussian data type, and the number of Gaussian data dimensions.

[0264] The Gaussian component ID is identification information for uniquely identifying a Gaussian component. The Gaussian data type indicates the type of the Gaussian component. For example, a value of 0 indicates "rotation," a value of 1 indicates "scale," a value of 2 indicates "SH coefficient," and a value of 3 indicates "transmittance." Note that the combinations of values ​​and types here are merely examples, and the combinations are not limited to these.

[0265] The Gaussian data dimension indicates the dimension of a Gaussian component (the number of subcomponents). For example, in the example shown in Fig. 42, the Gaussian data dimension of rotation is 4, the Gaussian data dimension of scale is 3, and the Gaussian data dimension of SH coefficients is 48.

[0266] The mapping information also includes a G-PCC component ID, a G-PCC dimension ID, and conversion information as information for each dimension of the Gaussian component. The G-PCC subcomponent ID corresponds to "ID" shown in FIG. 42. The G-PCC dimension ID is an identifier indicating the subcomponent number. For example, C1 included in the SH coefficients shown in FIG. 42 has a G-PCC dimension ID of 0, C2 has a G-PCC dimension ID of 1, and C3 has a G-PCC dimension ID of 2.

[0267] In this way, the mapping information indicates the G-PCC component ID and dimension ID corresponding to each dimension of each component of the Gaussian data. For example, for subcomponent C46 shown in Figure 42, a G-PCC component ID of 18 and a G-PCC dimension ID of 2 are indicated.

[0268] Note that Figure 43 shows an example of mapping information indicating the subcomponents (dimensions) of the G-PCC corresponding to each subcomponent (dimension) of the Gaussian data, but mapping information indicating the subcomponents of the Gaussian data corresponding to each subcomponent (dimension) of the G-PCC may also be used.

[0269] 43 shows an example of mapping information indicating the correspondence between subcomponents of Gaussian data and subcomponents of G-PCC, but if there is no restriction on the number of subcomponents in a G-PCC component, mapping information indicating the correspondence between Gaussian components and G-PCC components may be used. In other words, it is not necessary to indicate the correspondence between dimensions.

[0270] The conversion information indicates the parameters (conversion parameters) used when converting the component data. Fig. 44 shows an example of the configuration of the conversion information. For example, if a scale and an offset are used in the conversion, the conversion information includes a scale value and an offset value.

[0271] For example, the conversion is performed using the scale value and offset value according to the following formula:

[0272] Post-conversion data = pre-conversion data x scale value + offset

[0273] The formula used for the conversion may be a predefined formula, or multiple formulas may be defined and the formula to be used may be switched according to predetermined conditions. Furthermore, information indicating the formula used may be stored in the bitstream.

[0274] 43, the transformation information is provided for each subcomponent (dimension) of the Gaussian data, but the transformation information may be provided for each component of the Gaussian data. In other words, the transformation method may be changed for each component or for each subcomponent.

[0275] Note that some of the syntax shown in Fig. 43 may be omitted. For example, if the order of description of the information on rotation, scale, and SH coefficients and the number of dimensions are predetermined, the Gaussian data component ID, Gaussian data type, the number of Gaussian data dimensions, etc. may be omitted.

[0276] Furthermore, each piece of attribute information is encoded based on the mapping information to the attribute component determined by the second pre-processing unit 332. Fig. 45 is a diagram showing an example of the structure of encoded data when Gaussian data is G-PCC encoded.

[0277] In the SPS, for each G-PCC component, a G-PCC attribute component ID (attr_id), a G-PCC attribute type (attribute_type), and the number of dimensions of the G-PCC attribute component (num_dimension) are described. Here, a new G-PCC attribute type may be defined that indicates that each element of Gaussian data is to be coded.

[0278] 46 is a diagram showing an example of G-PCC attribute types for Gaussian data. As shown in FIG. 46, for example, G-PCC attribute types may be defined that respectively indicate the rotation, scale, SH coefficient, and transmittance of Gaussian data.

[0279] A G-PCC attribute component ID (attr_id) is assigned to the header of each ADU. A decoding device can identify the G-PCC attribute type of each ADU by using the G-PCC attribute component ID included in the ADU header and the information included in the SPS.

[0280] Furthermore, the mapping information (Gaussian data attribute mapping information SEI) may be transmitted as SEI or may be included in the SPS. For example, the SPS may include mapping information indicating Gaussian components corresponding to G-PCC components.

[0281] 47 is a flowchart of processing by the decoding device according to the present embodiment. First, the decoding device acquires a plurality of TLV units from a bit stream, and acquires a plurality of data units (GDU, ADU) and a plurality of parameter sets (SPS, GPS, APS) from the plurality of TLV units (S101).

[0282] Next, the decoding device decodes the three-dimensional coordinates using the SPS, GPS, and GDU (S102). Next, the decoding device decodes the multiple attribute components described in the SPS using the APS, ADU, and three-dimensional coordinates (S103).

[0283] Next, the decoding device analyzes the Gaussian data attribute mapping information SEI to obtain mapping information (S104). Next, the decoding device reconstructs second Gaussian data from the decoded attribute components and the mapping information (S105). Next, the decoding device obtains transformation information and generates first Gaussian data by inversely transforming the second Gaussian data using the transformation information (S106).

[0284] [Others] In the above description, for simplicity, an example in which Gaussian data is one frame is shown, but this method can also be applied to cases in which Gaussian data is present in multiple frames. In this case, multi-frame coding in the G-PCC method or inter-prediction in the G-PCC method may be used.

[0285] The encoding device may also generate a plurality of pieces of divided data by dividing the Gaussian data into a plurality of regions and encode each of the plurality of pieces of divided data. The encoding device may also assign the pieces of divided data to tiles or slices in the G-PCC and encode them.

[0286] In the above explanation, an example was shown in which attribute information (scale, rotation, SH coefficient, transmittance, etc.) of the second Gaussian data was encoded, but by using the method of this embodiment, it is also possible to encode other attribute information.

[0287] Fig. 48 is a block diagram showing a system for encoding and decoding first Gaussian data. The system shown in Fig. 48 includes a second pre-processing unit 371, a G-PCC encoding unit 372, a G-PCC decoding unit 373, and a second post-processing unit 374. Fig. 49 is a block diagram showing an example configuration of the second pre-processing unit 371. The second pre-processing unit 371 includes a conversion unit 375 and a mapping unit 376.

[0288] For example, when encoding a covariance matrix associated with three-dimensional coordinates included in the first Gaussian data, the conversion unit 375 included in the second pre-processing unit 371 converts the format of the covariance matrix. The mapping unit 376 generates converted first Gaussian data by mapping the format-converted covariance matrix to G-PCC attribute information. The G-PCC encoding unit 372 generates encoded data by encoding the converted first Gaussian data, the conversion information, and the mapping information.

[0289] The G-PCC decoding unit 373 decodes the encoded data to generate transformed first Gaussian data, transformation information, and mapping information. The second post-processing unit 374 uses the mapping information and transformation information to inversely transform the transformed first Gaussian data, thereby restoring the first Gaussian data. In other words, the covariance matrix is ​​restored. In this way, the system can encode and decode the first Gaussian data.

[0290] Furthermore, in converting the covariance matrix, the conversion unit 375 may convert a 3x3 covariance matrix into a format that can be encoded by G-PCC. For example, the conversion unit 375 divides the 3x3 covariance matrix into three three-dimensional components. Alternatively, if the covariance matrix is ​​a symmetric matrix, the conversion unit 375 may divide the six-dimensional elements, excluding elements having the same value, into two three-dimensional components. FIG. 50 is a diagram illustrating this conversion process. In the example shown in FIG. 50 , of the nine elements of the covariance matrix, six elements ([Sxx, Sxy, Sxz, Syy, Syz, Szz]) are extracted, excluding elements (Syz, Szx, Szy) having the same value as other elements.

[0291] Furthermore, conversion information indicating the details of the conversion is generated. The conversion information is stored in the metadata. Note that if the conversion method is uniquely determined in advance, the conversion information does not need to be stored in the coded data (bitstream).

[0292] As described above, by using the processes and methods described in this embodiment, it is possible to encode Gaussian data using G-PCC.

[0293] Specifically, the encoding device encodes three-dimensional coordinates included in the Gaussian data using position information encoding in G-PCC, and also encodes attribute information (scale, rotation, SH coefficient, transparency, etc.) associated with the three-dimensional coordinates using attribute information encoding in G-PCC.

[0294] The encoding device also converts each element of the attribute information of the Gaussian data into a data format that can be encoded by G-PCC. The encoding device also stores conversion information indicating the content of the conversion in metadata. This allows the decoding device to reconstruct the Gaussian data from the decoded data.

[0295] The encoding device also associates components or dimensions of Gaussian data with attribute components or dimensions that are units that can be coded using G-PCC. This allows the encoding device to code the Gaussian data using the G-PCC coding method. Furthermore, by defining a G-PCC attribute type that corresponds to the Gaussian data, the encoding device can code the Gaussian data using the G-PCC coding method.

[0296] The encoding device also stores mapping information in the metadata that indicates the correspondence between the components and dimensions of the Gaussian data and the attribute components and dimensions, allowing the decoding device to reconstruct the Gaussian data from the decoded data.

[0297] (Second Aspect) [Gaussian Data Encoding Method] Hereinafter, a general-purpose data structure for encoding Gaussian data and a data structure that enables Gaussian data to be encoded by G-PCC or other encoding methods will be described.

[0298] The following data structure provides a general-purpose method for encoding Gaussian data using various 3D encoding methods or 3D codec standards, allowing Gaussian data to be encoded using various 3D codec methods, including standards created by various standards organizations, international standards, national standards, and standards established by any organization.

[0299] 51 is a diagram showing a protocol stack of an encoding standard for encoding Gaussian data. 3D Gaussian codec is an encoding standard for encoding Gaussian data. Specific encoding methods for the 3D Gaussian codec include the V3C standard (ISO / IEC 23090-5), which encodes 3D data using one or more video codecs, the G-PCC standard (ISO-IEC 23090-9), which is a point cloud compression standard, and Draco, which is used for 3D mesh data. When the V3C standard is used, a video codec standard such as HEVC or VVC (Versatile Video Coding) is used.

[0300] 52 is a block diagram showing an example configuration of an encoding device 400 (three-dimensional data encoding device) according to this embodiment. The encoding device 400 generates a bit stream (encoded data) by encoding second Gaussian data. This encoding device 400 includes a second pre-processing unit 401, a G-PCC encoding unit 402, a V3C encoding unit 403, a metadata encoding unit 404, and a multiplexing unit 405.

[0301] The second pre-processing unit 401 converts and maps the second Gaussian data into data that can be encoded by the 3D codec used, depending on the 3D codec used.

[0302] For example, the second preprocessing unit 401 converts and maps the second Gaussian data into data that can be coded using the G-PCC coding method. The G-PCC coding unit 402 codes the converted data using the G-PCC coding method to generate a G-PCC TLV unit.

[0303] The second preprocessing unit 401 converts and maps the second Gaussian data into data that can be coded using the V3C coding method. The V3C coding unit 403 codes the converted data using the V3C coding method to generate a V3C unit.

[0304] The metadata encoding unit 404 generates metadata by encoding the transformation information and mapping information related to the transformation and mapping performed by the second pre-processing unit 401. For example, the metadata encoding unit 404 stores the transformation information and mapping information in the metadata.

[0305] The multiplexing unit 405 generates a bitstream by unitizing and multiplexing the G-PCC TLV unit or V3C unit and the metadata. Specifically, the multiplexing unit 405 stores the G-PCC TLV unit or V3C unit and the metadata in a 3D data unit, which is a data unit of a 3D Gaussian Codec Layer.

[0306] Fig. 53 is a block diagram showing an example configuration of a decoding device 410 (three-dimensional data decoding device) according to this embodiment. The decoding device 410 generates second Gaussian data by decoding a bit stream. This bit stream is generated, for example, by the encoding device 400 shown in Fig. 52. The decoding device 410 includes a demultiplexing unit 411, a G-PCC decoding unit 412, a V3C decoding unit 413, a metadata decoding unit 414, and a second post-processing unit 415.

[0307] The demultiplexing unit 411 demultiplexes and demultiplexes the 3D data units included in the bitstream to generate G-PCC TLV units or V3C units and metadata.

[0308] The G-PCC decoder 412 generates decoded data by decoding the G-PCC TLV unit using the G-PCC decoding method. The V3C decoder 413 generates decoded data by decoding the V3C unit using the V3C decoding method.

[0309] The metadata decoding unit 414 generates the conversion information and the mapping information by decoding the metadata. For example, the metadata decoding unit 414 obtains the conversion information and the mapping information from the metadata.

[0310] The second post-processing unit 415 inversely transforms the decoded data into second Gaussian data according to the 3D codec used for decoding.

[0311] 54 is a flowchart of the encoding process performed by the encoding device 400. First, the encoding device 400 determines the codec type to be used for encoding (S201). For example, the codec type to be used for encoding may be specified by the user, or may be determined based on the contents of the Gaussian data.

[0312] If the codec type used for encoding is the G-PCC encoding method (G-PCC in S201), the encoding device 400 encodes the Gaussian data using the G-PCC encoding method and outputs a G-PCC TLV unit (S202).

[0313] On the other hand, if the codec type used for encoding is the V3C encoding method (V3C in S201), the encoding device 400 encodes the Gaussian data using the V3C encoding method and outputs a V3C unit (S203).

[0314] Next, the encoding device 400 stores information indicating the codec type used for encoding in common metadata (S204). Here, the common metadata is metadata that can be used in common across multiple encoding methods (G-PCC encoding method and V3C encoding method).

[0315] Next, the encoding device 400 stores the conversion information and mapping information in the metadata (S205). Next, the encoding device 400 multiplexes the metadata and the encoding unit (G-PCC TLV unit or V3C unit) into a 3D data unit that can be commonly used in the G-PCC encoding method and the V3C encoding method (S206).

[0316] 55 is a flowchart of the decoding process by the decoding device 410. First, the decoding device 410 obtains a common 3D data unit, and then obtains data units and metadata from the 3D data unit (S211). Next, the decoding device 410 analyzes the metadata and determines the codec type used for encoding (the codec type to be used for decoding) based on information indicating the codec type included in the metadata (S212).

[0317] If the codec type used for encoding is the G-PCC encoding method (G-PCC in S213), the decoding device 410 acquires the TLV unit and decodes it using the G-PCC decoding method to generate a component (S214). On the other hand, if the codec type used for encoding is V3C (V3C in S213), the decoding device 410 acquires the V3C unit and decodes it using the V3C decoding method to generate a component (S215).

[0318] Next, the decoding device 410 reconstructs Gaussian data using the decoded components and the mapping information (S216). Next, the decoding device 410 obtains transformation information and generates second Gaussian data by inversely transforming the Gaussian data using the transformation information (S217).

[0319] 56 is a diagram showing the basic data structure of the Gaussian codec. The 3D data unit in the 3D Gaussian codec layer is a common unit in the Gaussian codec.

[0320] Fig. 57 is a diagram showing an example of the configuration of a 3D data unit (3DDataUnit). The payload (3DDU_data) of the 3D data unit stores various data units that make up the Gaussian codec. The header (3DDU_header) of the 3D data unit includes a 3D data unit type (3DDataUnitType) that indicates the type of data stored in the payload. Examples of the 3D data unit type (3DDataUnitType) include (1) a V3C unit, (2) a G-PCC TLV unit, (3) a Draco unit, (4) a 3D parameter set (3DPS) that is a parameter set common to multiple codecs, (5) a Gaussian sequence parameter set (GSSPS) that is a parameter set including metadata common to a sequence, (6) a Gaussian frame parameter set (GSFPS) that is a parameter set including metadata common to a frame (a 3D model at a certain time or time interval), and (7) an SEI (Gaussian GSSEI).

[0321] Furthermore, when the 3D data unit type (3DDataUnitType) is a V3C unit, the payload of the V3C unit includes an ACL_NAL unit or a VCL_NAL unit. The VCL NAL unit belongs to the video codec layer and is defined in a two-dimensional video compression standard such as HEVC or VVC. The ACL_NAL unit belongs to the atlas codec layer. The ACL_NAL unit stores data for restoring a two-dimensional image to three-dimensional data.

[0322] The 3D data unit may store a Draco unit based on a standard such as Draco.

[0323] Fig. 58 is a diagram showing a configuration example of a 3DPS (3DParameterset), which is a common parameter set in Gaussian coding. The 3DPS includes information (profile_level) indicating a profile and level that indicates a combination of an encoding standard, encoding method, and toolset used in encoding Gaussian data, and Gaussian component information (3D_codec_type). This information includes information (profile) indicating a profile, information (level) indicating a level, and information (3D_codec_type) indicating the encoding standard and encoding method.

[0324] 3D_codec_type indicates the encoding standard used for encoding Gaussian data. For example, 3D_codec_type=0 indicates that the V3C encoding method was used for encoding, and 3D_codec_type=1 indicates that the G-PCC encoding method was used for encoding.

[0325] A decoding device can determine a decoding method by analyzing 3D_codec_type. The decoding device may switch the decoding method based on 3D_codec_type, or may decide not to perform decoding if it does not support the decoding function of the encoding method indicated by 3D_codec_type. Note that the profile and level may include information indicating the encoding standard.

[0326] Furthermore, restrictions may be imposed on the 3D data unit type (3DDataUnitType) depending on the encoding method. For example, if the encoding method is G-PCC, the use of V3C units may be prohibited, and if the encoding method is V3C, the use of G-PCC TLV units may be prohibited.

[0327] The Gaussian data component information includes information indicating the configuration of a Gaussian component. Fig. 59 is a diagram showing an example of the configuration of the Gaussian data component information. The Gaussian data component information includes the number of Gaussian data components indicating the number of Gaussian components.

[0328] The Gaussian data component information includes, for each Gaussian component, a Gaussian component ID, a Gaussian data type, and the number of Gaussian data dimensions. The Gaussian component ID is identification information for uniquely identifying a Gaussian component. The Gaussian data type indicates the type of Gaussian component (rotation, scale, SH coefficient, transmittance, etc.). The number of Gaussian data dimensions indicates the number of dimensions of the Gaussian component (the number of subcomponents).

[0329] The Gaussian data component information also includes mapping information that depends on the codec. For example, if the codec type (3D_codec_type) is G-PCC, the Gaussian data component information includes G-PCC mapping information (gpcc_mapping_info) that indicates the correspondence between the Gaussian data components and the G-PCC codec components.

[0330] 60 is a diagram showing an example of the configuration of G-PCC mapping information. The G-PCC mapping information includes, as information for each dimension of a Gaussian component, a G-PCC component ID, a G-PCC dimension ID, and conversion information.

[0331] The G-PCC subcomponent ID is an identifier that indicates the G-PCC component number. The G-PCC dimension ID is an identifier that indicates the subcomponent number of the G-PCC component. The conversion information indicates the parameters (conversion parameters) used when converting the data of the dimension.

[0332] Furthermore, when the codec type (3D_codec_type) is V3C, the Gaussian data component information includes V3C mapping information that indicates the correspondence between the components of the Gaussian data and the components of the V3C codec.

[0333] 61 is a diagram showing an example of the configuration of V3C mapping information. The V3C mapping information includes, as information for each dimension of a Gaussian component, a V3C component ID, a V3C dimension ID, and conversion information.

[0334] The V3C subcomponent ID is an identifier indicating the V3C component number. The V3C dimension ID is an identifier indicating the subcomponent number of the V3C component. The conversion information indicates the parameters (conversion parameters) used when converting the data of the dimension.

[0335] The Gaussian data component information may include geometry component information in addition to attribute component information.

[0336] Moreover, although an example of mapping information indicating G-PCC components corresponding to Gaussian components has been shown here, mapping information indicating Gaussian components corresponding to G-PCC components may also be used. Moreover, although an example of mapping information indicating the correspondence between subcomponents of Gaussian data and subcomponents of G-PCC has been shown here, mapping information indicating the correspondence between Gaussian components and G-PCC components may also be used. In other words, the correspondence between subcomponents does not have to be shown.

[0337] Furthermore, some of the syntax shown here may be omitted. For example, if the order of description of the information on rotation, scale, and SH coefficients and the number of dimensions are predetermined, the Gaussian data component ID, Gaussian data type, and number of Gaussian data dimensions may be omitted.

[0338] Another example of a data unit will be described below. Instead of storing Gaussian data in either the V3C unit or the G-PCC TLV unit, a data unit specific to Gaussian data (Gaussian data unit) may be defined.

[0339] 62 is a diagram showing an example of the configuration of a Gaussian data unit. For example, the header of the Gaussian data unit includes a Gaussian data unit type (Gaussian DataUnitType) corresponding to the above-mentioned 3D data unit type (3DDataUnitType). The Gaussian data unit type indicates the type of data stored in the payload of the Gaussian data unit. For example, the Gaussian data unit type includes a geometry data unit and an attribute data unit. The payloads of the geometry data unit and the attribute data unit store encoded geometry data and attribute data, respectively.

[0340] The header of the attribute data unit includes an attribute data unit type. The attribute data unit type indicates the type of data stored in the payload of the attribute data unit. The attribute data unit type indicates an element of attribute information, and includes, for example, a rotation data unit, a scale data unit, a color coefficient data unit, an alpha data unit, etc. The payloads of the rotation data unit, scale data unit, color coefficient data unit, and alpha data unit store encoded rotation, scale, SH coefficient, and transparency, respectively.

[0341] Although the above description has been given of an example of encoding 3D Gaussian data, a similar technique may be applied to data other than 3D Gaussian data. For example, a similar technique may be applied to a 3D model or 3D data using AI.

[0342] Furthermore, in the above description, examples have been described in which the encoding methods are mainly V3C and G-PCC, but encoding methods other than V3C and G-PCC may also be used. Furthermore, the encoding method may be other than a 3D codec. For example, an image codec, a neural network codec, an audio codec, or the like may also be used as the encoding method.

[0343] 63 is a diagram showing a protocol stack of a standard for encoding a 3D Gaussian. As a method for encoding a 3D Gaussian, a protocol stack or a data structure for encoding using at least one of a video base codec and a geometry base codec may be used.

[0344] A video-based codec is image encoding that uses a video codec, such as standards such as AVC, HEVC, or VVC standardized by MPEG, or standards such as VPCC (Video-based Point Cloud Compression), VCM (Video Coding for Machines), or VDMC (Video-based Dynamic Mesh Compression).

[0345] Geometry-based codecs include point cloud compression standards such as G-PCC or Draco, or mesh data compression standards, etc. Note that 3D Gaussian coding methods may be extended to include neural network-based codecs such as Neural Network Coding (NNC).

[0346] Note that instead of the 3D Gaussian, a standard for encoding a 3D model may be defined. Fig. 64 is a diagram showing a protocol stack of a standard for encoding a 3D model. A protocol stack and a data structure for encoding a 3D model using at least one of V3C and G-PCC may be used.

[0347] The 3D model may include (1) a point cloud, (2) a mesh, (3) a 3D model created using NeRF (Neural Radiance Fields), and (4) a 3D model generated using Gaussian Splatting.

[0348] Furthermore, the encoding device may encode some of the components of the 3D model data using video encoding and other components using geometry-based encoding. For example, the encoding device may encode the three-dimensional coordinates, rotation, and scale, which are information representing shape information included in the Gaussian data, using G-PCC, and encode the SH coefficients using a video-based encoding method. Here, geometry-based encoding may be suitable for sparse geometric structure shapes, while video-based encoding may be suitable for multidimensional SH coefficients. Therefore, in such cases, encoding efficiency can be improved.

[0349]

[0111] Figure 65 is a diagram showing an example of the configuration of Gaussian component information in this case. The Gaussian component information shown in Figure 65 includes, in addition to the Gaussian component information shown in Figure 59, a 3D codec type (3D_codec_type) for each Gaussian component. This allows the 3D codec type to be set for each component, thereby realizing the above function. Note that, when a standard for encoding a 3D model as shown in Figure 64 is used, for example, instead of the Gaussian component information, similar information is stored in 3D model component information indicating information about the components of the 3D model.

[0350] [Summary] As described above, the encoding device (three-dimensional data encoding device) according to the embodiment performs the processing shown in FIG. 66. FIG. 66 is a flowchart of the encoding processing by the encoding device. The encoding device encodes Gaussian data. The Gaussian data includes a plurality of first components (e.g., a plurality of Gaussian components), each of which is expressed in one or more dimensions, and the plurality of first components includes a first component indicating position coordinates (e.g., three-dimensional point coordinates or two-dimensional coordinates) and a first component indicating information on a scale value based on the position coordinates. The encoding device generates encoded data by encoding the plurality of first components using a point cloud encoding method (e.g., G-PCC encoding method) (S301), and generates a bitstream including the encoded data (S302).

[0351] This allows the encoding device to encode Gaussian data using a point cloud encoding method, thereby enabling efficient processing since Gaussian data can be encoded without creating a new encoding method for Gaussian data.

[0352] For example, the plurality of first components may further include a first component indicating information on a rotation value based on a position coordinate. For example, the plurality of first components may further include a first component indicating color information or a first component indicating transmittance. For example, the rotation value and the scale value are expressed as information on a covariance matrix. For example, the plurality of first components may further include a first component indicating a coefficient of a spherical harmonic function.

[0353] For example, the encoding device further converts a plurality of first components (e.g., Gaussian components) into a plurality of second components (e.g., attribute components), and encodes the plurality of second components in the encoding (S301). In this way, for example, the encoding device can encode Gaussian data using the point cloud encoding method by converting data into a format compatible with the point cloud encoding method.

[0354] For example, the encoding device may further store information (e.g., mapping information) indicating a correspondence between the plurality of first components and the plurality of second components in metadata, and the bitstream may further include the metadata, allowing the decoding device to convert the plurality of second components into the plurality of first components using the information included in the bitstream.

[0355] For example, the number of dimensions of the plurality of second components is equal to or less than a predetermined number, and in the conversion, a first component among the plurality of first components whose number of dimensions is greater than the predetermined number is divided into two or more second components. With this, for example, the encoding device can encode Gaussian data using the point cloud encoding method by converting the data into a format compatible with the point cloud encoding method.

[0356] For example, in the encoding (S301), the encoding device encodes the first component indicating the position coordinates using a geometry encoding method included in the point cloud encoding method, and encodes the first component indicating the scale value information using an attribute encoding method included in the point cloud encoding method. This allows the encoding device to appropriately encode each piece of information using the geometry encoding method and the attribute encoding method included in the point cloud encoding method.

[0357] For example, the bitstream includes a parameter set including information indicating the types of the multiple first components, allowing the decoding device to determine the types of the multiple first components using the information included in the bitstream.

[0358] 67 is a block diagram of the encoding device 10. For example, the encoding device 10 includes a processor 11 and a memory 12, and the processor 11 performs the above-described processing using the memory 12.

[0359] Furthermore, a decoding device (three-dimensional data decoding device) according to an embodiment performs the processing shown in FIG. 68 . FIG. 68 is a flowchart of the decoding processing by the decoding device. The decoding device decodes Gaussian data. The Gaussian data includes multiple first components (e.g., multiple Gaussian components), each of which is expressed in one or more dimensions, and the multiple first components include a first component indicating position coordinates (e.g., three-dimensional point coordinates or two-dimensional coordinates) and a first component indicating information on a scale value based on the position coordinates. The decoding device acquires a bitstream including encoded data generated by encoding the multiple first components (S311), and generates the multiple first components by decoding the encoded data using a point cloud decoding method (e.g., a decoding method corresponding to the G-PCC encoding method) (S312).

[0360] This allows the encoding device to encode Gaussian data using a point cloud encoding method. Therefore, the Gaussian data can be encoded without creating a new encoding method for Gaussian data, thereby achieving efficient processing. Furthermore, the decoding device can appropriately decode the encoded data generated by the encoding device.

[0361] For example, the plurality of first components may further include a first component indicating information on a rotation value based on the position coordinates, or a first component indicating color information or a first component indicating transmittance.

[0362] For example, the rotation and scale values ​​are represented as information in a covariance matrix.For example, the plurality of first components further includes a first component representing a coefficient of a spherical harmonic function.

[0363] For example, the coded data is generated by encoding a plurality of second components (e.g., a plurality of attribute components) obtained by transforming a plurality of first components (e.g., a plurality of Gaussian components). In the decoding step (S312), the decoding device decodes the coded data using a point cloud decoding method to generate a plurality of second components, and transforms the plurality of second components into a plurality of first components.

[0364] According to this, for example, an encoding device can generate coded data by encoding Gaussian data using the point cloud coding method by converting data into a format compatible with the point cloud coding method, and a decoding device can appropriately decode the coded data generated by the encoding device.

[0365] For example, the decoding device may further obtain information (e.g., mapping information) indicating a correspondence between the first components and the second components from metadata included in the bitstream, thereby enabling the decoding device to convert the second components into the first components using the information included in the bitstream.

[0366] For example, the number of dimensions of the second components is equal to or less than a predetermined number. In the transform (S312), the decoding device combines two or more of the second components among the second components with a first component having a number of dimensions greater than the predetermined number.

[0367] This allows, for example, an encoding device to encode Gaussian data using the point cloud encoding method by converting the data into a format compatible with the point cloud encoding method, and a decoding device to appropriately decode the encoded data generated by the encoding device.

[0368] For example, in decoding (S312), the decoding device decodes the first component indicating the position coordinates using a geometry decoding method included in the point cloud decoding method, and decodes the first component indicating the scale value information using an attribute decoding method included in the point cloud decoding method.

[0369] This allows the encoding device to appropriately encode each piece of information using the geometry encoding method and attribute encoding method included in the point cloud encoding method, and the decoding device to appropriately decode the encoded data generated by the encoding device.

[0370] For example, the bitstream includes a parameter set including information indicating the types of the multiple first components, allowing the decoding device to determine the types of the multiple first components using the information included in the bitstream.

[0371] 69 is a block diagram of the decoding device 20. For example, the decoding device 20 includes a processor 21 and a memory 22, and the processor 21 uses the memory 22 to perform the above-described processing.

[0372] Moreover, an encoding device (three-dimensional data encoding device) according to an embodiment performs the processing shown in FIG. 70 . FIG. 70 is a flowchart of the encoding processing by the encoding device. The encoding device encodes Gaussian data. The Gaussian data includes a plurality of first components (e.g., a plurality of Gaussian components), and each of the plurality of first components is represented by one or more dimensions. The encoding device selects one of a plurality of encoding methods (S321), generates encoded data by encoding the plurality of first components using the selected encoding method (S322), and generates a bitstream including the encoded data and first information indicating the selected encoding method (e.g., a 3D codec type (3D_codec_type)) (S323).

[0373] This allows the encoding device to selectively select an encoding method to be used for encoding Gaussian data from a plurality of encoding methods, which may improve encoding efficiency.

[0374] For example, the encoding device further converts a plurality of first components (e.g., a plurality of Gaussian components) into a plurality of second components (e.g., attribute components) corresponding to the selected encoding method, and in encoding (S322), encodes the plurality of second components, and the bitstream further includes second information (e.g., mapping information) indicating the correspondence between the plurality of first components and the plurality of second components.

[0375] According to this, the encoding device can encode the Gaussian data using the selected encoding method by converting the data into a format corresponding to the selected encoding method, and the decoding device can convert multiple second components into multiple first components using second information included in the bitstream.

[0376] For example, the multiple first components include a first component indicating position coordinates (for example, three-dimensional point coordinates or two-dimensional coordinates) and a first component indicating information on a scale value based on the position coordinates.

[0377] For example, the plurality of coding schemes includes a geometry-based coding scheme and a video-based coding scheme. For example, the plurality of coding schemes further includes a coding scheme for a neural network. For example, the bitstream includes a parameter set common to the plurality of coding schemes, and the parameter set includes the first information.

[0378] For example, the bitstream includes multiple units common to multiple encoding methods, the first information includes multiple pieces of third information included in the headers of each of the multiple units, each of the multiple units stores either first encoded data encoded using a geometry-based encoding method, second encoded data encoded using a video-based encoding method, or multiple pieces of data including common parameters, and each of the multiple pieces of third information indicates the type of data stored in the unit containing the third information.

[0379] Furthermore, a decoding device (three-dimensional data decoding device) according to an embodiment performs the processing shown in FIG. 71 . FIG. 71 is a flowchart of the decoding processing by the decoding device. The decoding device decodes Gaussian data. The Gaussian data includes multiple first components (e.g., multiple Gaussian components), and each of the multiple first components is represented by one or more dimensions. The decoding device acquires a bitstream including encoded data generated by encoding the multiple first components (S331), acquires first information (e.g., a 3D codec type (3D_codec_type)) indicating an encoding method used to encode the multiple first components from among multiple encoding methods included in the bitstream (S332), and generates multiple first components by decoding the encoded data using a decoding method corresponding to the encoding method indicated by the first information (S333).

[0380] This allows the encoding device to selectively select an encoding method to be used for encoding Gaussian data from among multiple encoding methods, which may improve encoding efficiency, and also allows the decoding device to appropriately decode the encoded data generated by the encoding device.

[0381] For example, the coded data is generated by encoding a plurality of second components obtained by converting a plurality of first components. The decoding device further acquires second information (e.g., mapping information) indicating a correspondence relationship between the plurality of first components and the plurality of second components from the bitstream, and in the decoding (S333), generates a plurality of second components by decoding the coded data using a decoding method, and converts the plurality of second components into a plurality of first components using the correspondence relationship indicated by the second information.

[0382] According to this, the encoding device can encode the Gaussian data using the selected encoding method by converting the data into a format corresponding to the selected encoding method, and the decoding device can convert multiple second components into multiple first components using second information included in the bitstream.

[0383] For example, the multiple first components include a first component indicating position coordinates (for example, three-dimensional point coordinates or two-dimensional coordinates) and a first component indicating information on a scale value based on the position coordinates.

[0384] For example, the plurality of coding schemes includes a geometry-based coding scheme and a video-based coding scheme. For example, the plurality of coding schemes further includes a coding scheme for a neural network. For example, the bitstream includes a parameter set common to the plurality of coding schemes, and the parameter set includes the first information.

[0385] For example, the bitstream includes multiple units common to multiple encoding methods, the first information includes multiple pieces of third information included in the headers of each of the multiple units, each of the multiple units stores either first encoded data encoded using a geometry-based encoding method, second encoded data encoded using a video-based encoding method, or multiple pieces of data including common parameters, and each of the multiple pieces of third information indicates the type of data stored in the unit containing the third information.

[0386] The above describes the encoding device (three-dimensional data encoding device) and decoding device (three-dimensional data decoding device) according to the embodiments and modifications of the present disclosure, but the present disclosure is not limited to these embodiments.

[0387] Furthermore, each processing unit included in the encoding device, decoding device, etc. according to the above-described embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0388] Furthermore, the integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI may also be used.

[0389] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0390] Furthermore, the present disclosure may be realized as an encoding method (three-dimensional data encoding method) or a decoding method (three-dimensional data decoding method) executed by an encoding device (three-dimensional data encoding device) and a decoding device (three-dimensional data decoding device), etc.

[0391] The present disclosure may also be realized as a program that causes a computer, processor, or device to execute the encoding method or decoding method. The present disclosure may also be realized as a bitstream generated by the encoding method. The present disclosure may also be realized as a recording medium on which the program or the bitstream is recorded. For example, the present disclosure may also be realized as a non-transitory computer-readable recording medium on which the program or the bitstream is recorded.

[0392] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0393] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0394] While the encoding device and decoding device according to one or more aspects have been described based on the embodiments, the present disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the present embodiments and configurations constructed by combining components of different embodiments may also be included within the scope of one or more aspects.

[0395] The present disclosure is applicable to encoding devices and decoding devices.

[0396] 10 Encoding device 11, 21 Processor 12, 22 Memory 20 Decoding device 101 Three-dimensional data encoding system 102 Three-dimensional data decoding system 103 Sensor terminal 104 External connection unit 111 Three-dimensional data generation system 112 Presentation unit 113 Encoding unit 114 Multiplexing unit 115 Input / output unit 116 Control unit 117 Sensor information acquisition unit 118 Three-dimensional data generation unit 121 Sensor information acquisition unit 122 Input / output unit 123 Demultiplexing unit 124 Decoding unit 125 Presentation unit 126 User interface 127 Control unit 130 First encoding unit 131 Position information encoding unit 132 Attribute information encoding unit 133 Additional information encoding unit 134 Multiplexing unit 140 First decoding unit 141 Demultiplexing unit 142 Position information decoding unit 143 Attribute information decoding unit 144 Additional information decoding unit 150 Second encoding unit 151 Additional information generation unit 152 Position image generation unit 153 Attribute image generation unit 154 Video encoding unit 155 Additional information encoding unit 156 Multiplexing unit 160 Second decoding unit 161 Demultiplexing unit 162 Video decoding unit 163 Additional information decoding unit 164 Position information generation unit 165 Attribute information generation unit 171 Octree encoding unit 172 Prediction tree encoding unit 173 Octree decoding unit 174 Prediction tree decoding unit 181, 191 Octree generation unit 182, 192 Geometric information calculation unit 183, 193 Encoding table selection unit 184 Entropy encoding unit 194 Entropy decoding unit 201 LoD attribute information encoding unit 202 Transformed attribute information encoding unit 203 LoD attribute information decoding unit 204 Transformed attribute information decoding unit 211 Sorting unit 212 Haar transform unit 213 Quantization unit 214, 222 Inverse quantization unit 215, 223 Inverse Haar transform unit 216, 224 Memory 217 Arithmetic coding unit 221 Arithmetic decoding unit 231 Data division unit 232 Encoding unit 233 Decoding unit 234 Data combination unit 241 Encoding unit 242 TLV storage unit 301 Sensor information input unit 302 Three-dimensional data generation unit 303, 305 Rendering unit 304 Spherical harmonic function 311 Encoding unit312 Multiplexing unit 321 Demultiplexing unit 322 Decoding unit 323 Application unit 324 Input interface unit 325 Rendering unit 326 Presentation unit 330, 400 Encoding device 331 First pre-processing unit 332, 371, 401 Second pre-processing unit 333, 372, 402 G-PCC encoding unit 341 Position information encoding unit 342 Rotation encoding unit 343 Scale encoding unit 344 SH coefficient encoding unit 345 Transmittance encoding unit 346 Metadata encoding unit 347, 405 Multiplexing unit 350, 410 Decoding device 351, 373, 412 G-PCC decoding unit 352, 374, 415 Second post-processing unit 353 First post-processing unit 361, 411 Demultiplexing unit 362 Position information decoding unit 363 Rotation decoding unit 364 Scale decoding unit 365 SH coefficient decoding unit 366 Transmittance decoding unit 367 Metadata decoding unit 375 Conversion unit 376 Mapping unit 403 V3C encoding unit 404 Metadata encoding unit 413 V3C decoding unit 414 Metadata decoding unit

Claims

1. A three-dimensional data encoding method for encoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of the plurality of first components being expressed in one or more dimensions, the three-dimensional data encoding method comprising: selecting one of a plurality of encoding methods; generating encoded data by encoding the plurality of first components using the selected encoding method; and generating a bitstream including the encoded data and first information indicating the selected encoding method.

2. The three-dimensional data encoding method according to claim 1, further comprising: converting the plurality of first components into a plurality of second components corresponding to the selected encoding method; encoding the plurality of second components; and the bit stream further including second information indicating a correspondence between the plurality of first components and the plurality of second components.

3. The three-dimensional data encoding method according to claim 1, wherein the plurality of first components include a first component indicating a position coordinate and a first component indicating information on a scale value based on the position coordinate.

4. The three-dimensional data encoding method according to claim 1, wherein said plurality of encoding methods includes a geometry-based encoding method and a video-based encoding method.

5. The three-dimensional data encoding method according to claim 4, wherein said plurality of encoding methods further includes an encoding method for a neural network.

6. The three-dimensional data encoding method according to claim 4, wherein the bit stream includes a parameter set common to the plurality of encoding methods, and the parameter set includes the first information.

7. The three-dimensional data encoding method according to claim 6, wherein the bitstream includes a plurality of units common to the plurality of encoding methods, the first information includes a plurality of pieces of third information included in the headers of each of the plurality of units, each of the plurality of units stores either first encoded data encoded using the geometry-based encoding method, second encoded data encoded using the video-based encoding method, or a plurality of data including the common parameters, and each of the plurality of pieces of third information indicates the type of data stored in the unit containing the third information.

8. A three-dimensional data decoding method for decoding Gaussian data, wherein the Gaussian data includes a plurality of first components, each of the plurality of first components being represented in one or more dimensions, the three-dimensional data decoding method comprising: acquiring a bit stream including encoded data generated by encoding the plurality of first components; acquiring first information indicating an encoding method used to encode the plurality of first components from a plurality of encoding methods included in the bit stream; and generating the plurality of first components by decoding the encoded data using a decoding method corresponding to the encoding method indicated by the first information.

9. A three-dimensional data decoding method as described in claim 8, wherein the encoded data is generated by encoding a plurality of second components into which the plurality of first components are converted, and the three-dimensional data decoding method further comprises: obtaining second information from the bit stream indicating a correspondence between the plurality of first components and the plurality of second components; and in the decoding, generating the plurality of second components by decoding the encoded data using the decoding method; and converting the plurality of second components into the plurality of first components using the correspondence indicated by the second information.

10. The three-dimensional data decoding method according to claim 8, wherein the plurality of first components include a first component indicating a position coordinate and a first component indicating information on a scale value based on the position coordinate.

11. The three-dimensional data decoding method according to claim 8, wherein the plurality of encoding methods includes a geometry-based encoding method and a video-based encoding method.

12. The three-dimensional data decoding method according to claim 11, wherein the plurality of encoding methods further includes an encoding method for a neural network.

13. The three-dimensional data decoding method according to claim 11, wherein the bit stream includes a parameter set common to the plurality of encoding methods, and the parameter set includes the first information.

14. The three-dimensional data decoding method according to claim 13, wherein the bitstream includes a plurality of units common to the plurality of encoding methods, the first information includes a plurality of pieces of third information included in the header of each of the plurality of units, each of the plurality of units stores either first encoded data encoded using the geometry-based encoding method, second encoded data encoded using the video-based encoding method, or a plurality of data including the common parameters, and each of the plurality of pieces of third information indicates the type of data stored in the unit containing the third information.

15. A three-dimensional data encoding device that encodes Gaussian data, comprising: a processor; and a memory; the Gaussian data includes a plurality of first components, each of the plurality of first components being expressed in one or more dimensions; the processor uses the memory to select one of a plurality of encoding methods; generate encoded data by encoding the plurality of first components using the selected encoding method; and generate a bitstream including the encoded data and first information indicating the selected encoding method.

16. A three-dimensional data decoding device that decodes Gaussian data, comprising: a processor; and a memory; the Gaussian data includes a plurality of first components, each of which is expressed in one or more dimensions; the processor uses the memory to obtain a bit stream including encoded data generated by encoding the plurality of first components; obtain first information, from a plurality of encoding methods included in the bit stream, indicating an encoding method used to encode the plurality of first components; and generate the plurality of first components by decoding the encoded data using a decoding method corresponding to the encoding method indicated by the first information.

Images