Three dimensional data conversion method, three dimensional data inverse conversion method, three dimensional data conversion device, and three dimensional data inverse conversion device

Abstract

To improve encoding efficiency.SOLUTION: A three dimensional data conversion method includes calculating a plurality of converted attribute values by adding an offset to an attribute value before conversion of each of a plurality of three dimensional points, each of the plurality of converted attribute values being an unsigned integer compressible by an encoder conforming to a predetermined format of the attribute value, and transmitting the offset to the encoder. For example, the offset may be a fixed value regardless of a plurality of attribute values before conversion. For example, the offset may be a minimum value among a plurality of attribute values before conversion.SELECTED DRAWING: Figure 33

Description

[Technical Field]

[0001] The present disclosure relates to a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, and a three-dimensional data decoding device. [Background technology]

[0002] In the future, devices and services that utilize 3D data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, video distribution, etc. 3D data can be acquired in a variety of ways, including distance sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

[0003] One method of representing three-dimensional data is a point cloud, which represents the shape of a three-dimensional structure using a group of points in three-dimensional space. A point cloud stores the position and color of the points. Point clouds are expected to become the mainstream method of representing three-dimensional data, but point clouds require a very large amount of data. Therefore, when storing or transmitting three-dimensional data, data compression through encoding is essential, just as with two-dimensional video images (examples include MPEG-4 AVC or HEVC standardized by MPEG).

[0004] In addition, compression of point clouds is partially supported by public libraries that perform point cloud-related processing (Point Cloud Library).

[0005] Furthermore, a technique is known in which three-dimensional map data is used to search for and display facilities located around a vehicle (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2014 / 020663 Summary of the Invention [Problem to be solved by the invention]

[0007] In such a three-dimensional data encoding method and three-dimensional data decoding method, it is desirable to be able to improve the encoding efficiency.

[0008] An object of the present disclosure is to provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency. [Means for solving the problem]

[0009] A three-dimensional data conversion method according to one aspect of the present disclosure calculates a plurality of converted attribute values ​​by adding an offset to the pre-conversion attribute value of each of a plurality of three-dimensional points, each of the plurality of converted attribute values ​​being an unsigned integer that can be compressed by an encoder that conforms to a predetermined format for attribute values, and transmits the offset to the encoder.

[0010] A three-dimensional data inverse conversion method according to one aspect of the present disclosure obtains, from a decoder, multiple post-conversion attribute values ​​and offsets of multiple three-dimensional points, each of the multiple post-conversion attribute values ​​being an unsigned integer that can be expanded by the decoder in accordance with a predetermined attribute value format, and subtracts the offsets from the multiple post-conversion attribute values ​​to calculate multiple pre-conversion attribute values ​​of the multiple three-dimensional points. [Effects of the Invention]

[0011] The present disclosure can provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing a configuration of a three-dimensional data encoding / decoding system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of point cloud data according to the first embodiment. [Figure 3] FIG. 3 is a diagram showing an example of the structure of a data file in which point cloud data information according to the first embodiment is described. [Figure 4] FIG. 4 is a diagram showing types of point cloud data according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing a configuration of a first encoding unit according to the first embodiment. [Figure 6] FIG. 6 is a block diagram of a first encoding unit according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating a configuration of a first decoding unit according to the first embodiment. [Figure 8] FIG. 8 is a block diagram of a first decoding unit according to the first embodiment. [Figure 9] FIG. 9 is a block diagram of a three-dimensional data encoding device according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of location information according to the first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of an octree representation of position information according to the first embodiment. [Figure 12] FIG. 12 is a block diagram of a three-dimensional data decoding device according to the first embodiment. [Figure 13] FIG. 13 is a block diagram of the attribute information encoding unit according to the first embodiment. [Figure 14] FIG. 14 is a block diagram of the attribute information decoding unit according to the first embodiment. [Figure 15] FIG. 15 is a block diagram showing a configuration of an attribute information encoding unit according to the first embodiment. [Figure 16] FIG. 16 is a block diagram of the attribute information encoding unit according to the first embodiment. [Figure 17] FIG. 17 is a block diagram showing a configuration of an attribute information decoding unit according to the first embodiment. [Figure 18] FIG. 18 is a block diagram of the attribute information decoding unit according to the first embodiment. [Figure 19] FIG. 19 is a diagram showing a configuration of a second encoding unit according to the first embodiment. [Figure 20] FIG. 20 is a block diagram of a second encoding unit according to the first embodiment. [Figure 21] FIG. 21 is a diagram illustrating a configuration of a second decoding unit according to the first embodiment. [Figure 22] FIG. 22 is a block diagram of a second decoding unit according to the first embodiment. [Figure 23] FIG. 23 is a diagram illustrating a protocol stack related to PCC encoded data according to the first embodiment. [Figure 24] FIG. 24 is a diagram illustrating the configurations of an encoding unit and a multiplexing unit according to the second embodiment. [Figure 25] FIG. 25 is a diagram illustrating an example of a structure of coded data according to the second embodiment. [Figure 26] FIG. 26 is a diagram illustrating an example of the structure of coded data and NAL units according to the second embodiment. [Figure 27] FIG. 27 is a diagram illustrating an example of the semantics of pcc_nal_unit_type according to the second embodiment. [Figure 28] FIG. 28 is a diagram showing an example of the transmission order of NAL units according to the second embodiment. [Figure 29] FIG. 29 is a flowchart of processing by the three-dimensional data encoding device according to the second embodiment. [Figure 30] FIG. 30 is a flowchart of processing by the three-dimensional data decoding device according to the second embodiment. [Figure 31] FIG. 31 is a flowchart of the multiplexing process according to the second embodiment. [Figure 32] FIG. 32 is a flowchart of the demultiplexing process according to the second embodiment. [Figure 33] FIG. 33 is a block diagram showing a configuration of a three-dimensional data encoding device according to the third embodiment. [Figure 34] FIG. 34 is a block diagram showing a configuration of a three-dimensional data decoding device according to the third embodiment. [Figure 35] FIG. 35 is a diagram illustrating a first example of the syntax of an SPS according to the third embodiment. [Figure 36] FIG. 36 is a diagram illustrating an example of a configuration of a bitstream according to the third embodiment. [Figure 37] FIG. 37 is a diagram illustrating a second example of the syntax of the SPS according to the third embodiment. [Figure 38] FIG. 38 is a diagram illustrating a first example of the syntax of the conversion information according to the third embodiment. [Figure 39] FIG. 39 is a diagram illustrating a second example of the syntax of the conversion information according to the third embodiment. [Figure 40] FIG. 40 is a flowchart showing the processing procedure of the three-dimensional data encoding device according to the third embodiment. [Figure 41] FIG. 41 is a flowchart showing the processing procedure of the three-dimensional data decoding device according to the third embodiment. [Figure 42] FIG. 42 is a block diagram illustrating another example of the processing performed by the three-dimensional data encoding device according to the third embodiment. [Figure 43] FIG. 43 is a block diagram illustrating another example of the processing performed by the three-dimensional data decoding device according to the third embodiment. [Figure 44] FIG. 44 is a diagram illustrating an example of syntax of the SEI according to the fourth embodiment. [Figure 45] FIG. 45 is a diagram illustrating an example of the syntax of ply_format_info() according to the fourth embodiment. [Figure 46] FIG. 46 is a diagram illustrating an example of the syntax of las_format_info() according to the fourth embodiment. [Figure 47] FIG. 47 is a diagram illustrating an example of the syntax of public_header_block() according to the fourth embodiment. [Figure 48] FIG. 48 is a diagram illustrating an example of the syntax of variable_length_records() according to the fourth embodiment. [Figure 49]FIG. 49 is a diagram illustrating an example of the syntax of point_data_records() according to the fourth embodiment. [Figure 50] FIG. 50 is a diagram illustrating an example of the syntax of extended_variable_length_records() according to the fourth embodiment. [Figure 51] FIG. 51 is a block diagram of a three-dimensional data encoding device according to a first example of the fourth embodiment. [Figure 52] FIG. 52 is a block diagram of a three-dimensional data decoding device according to a first example of the fourth embodiment. [Figure 53] FIG. 53 is a diagram illustrating an example of processing by a conversion unit according to the first example of the fourth embodiment. [Figure 54] FIG. 54 is a diagram illustrating an example of processing by a conversion unit according to the first example of the fourth embodiment. [Figure 55] FIG. 55 is a diagram illustrating an example of processing by an inverse transform unit according to the first example of the fourth embodiment. [Figure 56] FIG. 56 is a diagram illustrating an example of processing by an inverse transform unit according to the first example of the fourth embodiment. [Figure 57] FIG. 57 is a diagram illustrating an example of the syntax of an SPS according to the fourth embodiment. [Figure 58] FIG. 58 is a diagram illustrating an example of the syntax of attribute_parameter(i) according to the fourth embodiment. [Figure 59] FIG. 59 is a flowchart of three-dimensional data encoding processing according to the fourth embodiment. [Figure 60] FIG. 60 is a flowchart of three-dimensional data decoding processing according to the fourth embodiment. [Figure 61] FIG. 61 is a block diagram of a three-dimensional data encoding device according to a second example of the fourth embodiment. [Figure 62] FIG. 62 is a diagram illustrating an example of processing by a conversion unit according to the second example of the fourth embodiment. [Figure 63] FIG. 63 is a diagram illustrating an example of processing by an inverse transform unit according to the second example of the fourth embodiment. [Figure 64] FIG. 64 is a block diagram showing a configuration of a three-dimensional data encoding device according to a third example of the fourth embodiment. [Figure 65] FIG. 65 is a diagram illustrating an example of processing by a conversion unit according to the third example of the fourth embodiment. [Figure 66] FIG. 66 is a diagram illustrating an example of processing by a conversion unit according to the third example of the fourth embodiment. [Figure 67] FIG. 67 is a diagram illustrating an example of processing by a conversion unit according to the third example of the fourth embodiment. [Figure 68] FIG. 68 is a diagram illustrating an example of processing by an inverse transform unit according to the third example of the fourth embodiment. [Figure 69] FIG. 69 is a diagram illustrating an example of processing by an inverse transform unit according to the third example of the fourth embodiment. [Figure 70] FIG. 70 is a diagram illustrating an example of processing by an inverse transform unit according to the third example of the fourth embodiment. [Figure 71] FIG. 71 is a block diagram of a three-dimensional data encoding device according to a fourth example of the fourth embodiment. [Figure 72] FIG. 72 is a block diagram of a three-dimensional data decoding device according to a fourth example of the fourth embodiment. [Figure 73] FIG. 73 is a diagram illustrating an example of processing by a conversion unit according to the fifth example of the fourth embodiment. [Figure 74] FIG. 74 is a diagram illustrating an example of the syntax of the SPS and SEI according to the fourth embodiment. [Figure 75] FIG. 75 is a diagram showing an example of how conversion information according to the fourth embodiment is stored. [Figure 76] FIG. 76 is a flowchart of three-dimensional data encoding processing according to the fourth embodiment. [Figure 77] FIG. 77 is a flowchart of three-dimensional data decoding processing according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] A three-dimensional data encoding method according to one embodiment of the present disclosure transforms attribute information of three-dimensional points in at least one frame among a plurality of frames constituting a sequence, and generates a bitstream by encoding the transformed attribute information, wherein the bitstream further includes at least one first parameter of the transformation provided for the sequence, and at least one second parameter of the transformation provided for each of the at least one frame.

[0014] According to this, the three-dimensional data coding method can selectively control the switching of transformation parameters both on a sequence-by-sequence basis and on a frame-by-frame basis, thereby enabling appropriate transformation processing and improving coding efficiency.

[0015] For example, in a three-dimensional data decoding device that decodes the attribute information from the bitstream, if the bitstream includes the second parameter corresponding to the frame to be processed, the second parameter may be used.

[0016] For example, in the three-dimensional data decoding device, if the bitstream does not include the second parameters corresponding to the frame to be processed, the first parameters may be used.

[0017] For example, the conversion may involve multiplying or dividing the attribute information by a first value and / or adding or subtracting a second value, and each of the at least one first parameter and the at least one second parameter may indicate at least one of the first value or the second value.

[0018] For example, the bitstream may include multiple types of information about the three-dimensional points, including the attribute information, and the bitstream may further include first information indicating whether each of the multiple types of information is to be compressed.

[0019] For example, the plurality of types of information may include a plurality of types of attribute information of the three-dimensional points. For example, the plurality of types of information may include position information of the three-dimensional points.

[0020] For example, the bitstream may include first control information for the sequence, the first control information including information indicating a list of the multiple types of information, and the first information including information identifying the first control information and information indicating the number of the information to be compressed in the list.

[0021] For example, the bitstream may further include second information indicating a format type of the point cloud data including the attribute information.

[0022] A three-dimensional data decoding method according to one embodiment of the present disclosure generates decoded attribute information by decoding a bitstream, and generates attribute information of three-dimensional points of at least one frame among a plurality of frames constituting a sequence by inverse transforming the decoded attribute information, wherein the bitstream further includes at least one first parameter of the inverse transform provided for the sequence, and at least one second parameter of the inverse transform provided for each of the at least one frame.

[0023] According to this, the three-dimensional data decoding method can decode attribute information from a bitstream with improved coding efficiency.

[0024] For example, when the bitstream includes the second parameter corresponding to the frame to be processed, the decoding attribute information of the frame to be processed may be inversely converted using the second parameter.

[0025] For example, if the bitstream does not include the second parameter corresponding to the frame to be processed, the decoding attribute information of the frame to be processed may be inversely converted using the first parameter.

[0026] For example, in the inverse transformation, the decoded attribute information may be multiplied or divided by a first value and / or added or subtracted by a second value, and each of the at least one first parameter and the at least one second parameter may indicate at least one of the first value or the second value.

[0027] For example, the bitstream may include multiple types of information about the three-dimensional points, including the attribute information, and the bitstream may further include first information indicating whether each of the multiple types of information is to be compressed.

[0028] For example, the plurality of types of information may include a plurality of types of attribute information of the three-dimensional points. For example, the plurality of types of information may include position information of the three-dimensional points.

[0029] For example, the bitstream may include first control information for the sequence, the first control information including information indicating a list of the multiple types of information, and the first information including information identifying the first control information and information indicating the number of the information to be compressed in the list.

[0030] For example, the bitstream may further include second information indicating a format type of the point cloud data including the attribute information.

[0031] Furthermore, a three-dimensional data encoding device according to one embodiment of the present disclosure includes a processor and a memory, wherein the processor uses the memory to transform attribute information of three-dimensional points of at least one frame among a plurality of frames constituting a sequence, and generates a bitstream by encoding the transformed attribute information, and the bitstream further includes at least one first parameter of the transformation provided for the sequence, and at least one second parameter of the transformation provided for each of the at least one frame.

[0032] This allows the three-dimensional data encoding device to selectively control the switching of transformation parameters both on a sequence-by-sequence basis and on a frame-by-frame basis, thereby enabling appropriate transformation processing and improving encoding efficiency.

[0033] Furthermore, a three-dimensional data decoding device according to one embodiment of the present disclosure includes a processor and a memory, wherein the processor uses the memory to generate decoded attribute information by decoding a bitstream, and to generate attribute information of three-dimensional points of at least one frame among a plurality of frames constituting a sequence by inverse transforming the decoded attribute information, and the bitstream further includes at least one first parameter of the inverse transform provided for the sequence, and at least one second parameter of the inverse transform provided for each of the at least one frame.

[0034] This allows the three-dimensional data decoding device to decode attribute information from a bitstream with improved coding efficiency.

[0035] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0036] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0037] (Embodiment 1) When using encoded point cloud data in an actual device or service, it is desirable to transmit and receive the information required for the application in order to reduce network bandwidth. However, until now, such a function has not existed in the encoding structure of 3D data, and no encoding method for this purpose has existed.

[0038] In this embodiment, we will describe a three-dimensional data encoding method and a three-dimensional data encoding device that provide the function of transmitting and receiving information required for the purpose in encoded data of a three-dimensional point cloud, as well as a three-dimensional data decoding method and a three-dimensional data decoding device that decodes the encoded data, a three-dimensional data multiplexing method that multiplexes the encoded data, and a three-dimensional data transmission method that transmits the encoded data.

[0039] In particular, two encoding methods (encoding schemes) are currently being considered as encoding methods (encoding systems) for point cloud data; however, the structure of the encoded data and the method for storing the encoded data in a system format have not been defined, and as things stand, there is a problem that MUX processing (multiplexing) in the encoding unit, or transmission or storage, is not possible.

[0040] Furthermore, there has been no method to date that supports a format in which two codecs, a first encoding method and a second encoding method, are mixed, such as PCC (Point Cloud Compression).

[0041] In this embodiment, a description will be given of the structure of PCC encoded data in which two codecs, a first encoding method and a second encoding method, are mixed, and a method of storing the encoded data in a system format.

[0042] First, the configuration of a three-dimensional data (point cloud data) encoding / decoding system according to this embodiment will be described. Fig. 1 is a diagram showing an example of the configuration of a three-dimensional data encoding / decoding system according to this embodiment. As shown in Fig. 1, the three-dimensional data encoding / decoding system includes a three-dimensional data encoding system 4601, a three-dimensional data decoding system 4602, a sensor terminal 4603, and an external connection unit 4604.

[0043] The three-dimensional data encoding system 4601 generates encoded data or multiplexed data by encoding point cloud data, which is three-dimensional data. Note that the three-dimensional data encoding system 4601 may be a three-dimensional data encoding device realized by a single device, or may be a system realized by multiple devices. Furthermore, the three-dimensional data encoding device may include some of the multiple processing units included in the three-dimensional data encoding system 4601.

[0044] The three-dimensional data encoding system 4601 includes a point cloud data generation system 4611, a presentation unit 4612, an encoding unit 4613, a multiplexing unit 4614, an input / output unit 4615, and a control unit 4616. The point cloud data generation system 4611 includes a sensor information acquisition unit 4617 and a point cloud data generation unit 4618.

[0045] The sensor information acquisition unit 4617 acquires sensor information from the sensor terminal 4603 and outputs the sensor information to the point cloud data generation unit 4618. The point cloud data generation unit 4618 generates point cloud data from the sensor information and outputs the point cloud data to the encoding unit 4613.

[0046] The presentation unit 4612 presents the sensor information or point cloud data to the user. For example, the presentation unit 4612 displays information or an image based on the sensor information or point cloud data.

[0047] The encoding unit 4613 encodes (compresses) the point cloud data, and outputs the obtained encoded data, control information obtained in the encoding process, and other additional information to the multiplexing unit 4614. The additional information includes, for example, sensor information.

[0048] The multiplexing unit 4614 generates multiplexed data by multiplexing the coded data input from the coding unit 4613, the control information, and the additional information. The format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.

[0049] The input / output unit 4615 (e.g., a communication unit or an interface) outputs the multiplexed data to the outside. Alternatively, the multiplexed data is stored in a storage unit such as an internal memory. The control unit 4616 (or application execution unit) controls each processing unit. In other words, the control unit 4616 controls encoding, multiplexing, etc.

[0050] The sensor information may be input to the encoding unit 4613 or the multiplexing unit 4614. The input / output unit 4615 may output the point cloud data or the encoded data directly to the outside.

[0051] The transmission signal (multiplexed data) output from the three-dimensional data encoding system 4601 is input to the three-dimensional data decoding system 4602 via the external connection unit 4604 .

[0052] The three-dimensional data decoding system 4602 generates point cloud data, which is three-dimensional data, by decoding the encoded data or multiplexed data. Note that the three-dimensional data decoding system 4602 may be a three-dimensional data decoding device realized by a single device, or may be a system realized by multiple devices. Furthermore, the three-dimensional data decoding device may include some of the multiple processing units included in the three-dimensional data decoding system 4602.

[0053] The three-dimensional data decoding system 4602 includes a sensor information acquisition unit 4621 , an input / output unit 4622 , a demultiplexing unit 4623 , a decoding unit 4624 , a presentation unit 4625 , a user interface 4626 , and a control unit 4627 .

[0054] The sensor information acquisition unit 4621 acquires sensor information from the sensor terminal 4603 .

[0055] The input / output unit 4622 acquires the transmission signal, decodes the multiplexed data (file format or packets) from the transmission signal, and outputs the multiplexed data to the demultiplexer 4623.

[0056] The demultiplexing unit 4623 obtains the coded data, control information, and additional information from the multiplexed data, and outputs the coded data, control information, and additional information to the decoding unit 4624.

[0057] The decoding unit 4624 reconstructs the point cloud data by decoding the encoded data.

[0058] The presentation unit 4625 presents the point cloud data to the user. For example, the presentation unit 4625 displays information or images based on the point cloud data. The user interface 4626 acquires instructions based on user operations. The control unit 4627 (or the application execution unit) controls each processing unit. In other words, the control unit 4627 controls demultiplexing, decoding, presentation, etc.

[0059] The input / output unit 4622 may acquire point cloud data or encoded data directly from the outside. The presentation unit 4625 may acquire additional information such as sensor information and present information based on the additional information. The presentation unit 4625 may perform presentation based on a user instruction acquired by the user interface 4626.

[0060] The sensor terminal 4603 generates sensor information, which is information obtained by a sensor. The sensor terminal 4603 is a terminal equipped with a sensor or a camera, and examples thereof include a moving body such as an automobile, a flying object such as an airplane, a mobile terminal, or a camera.

[0061] The sensor information that can be acquired by the sensor terminal 4603 includes, for example, (1) the distance between the sensor terminal 4603 and an object, or the reflectance of the object, obtained from a LIDAR, millimeter-wave radar, or infrared sensor, and (2) the distance between a camera and an object, or the reflectance of the object, obtained from multiple monocular camera images or stereo camera images. The sensor information may also include the attitude, direction, gyro (angular velocity), position (GPS information or altitude), speed, acceleration, etc. of the sensor. The sensor information may also include temperature, air pressure, humidity, magnetism, etc.

[0062] The external connection unit 4604 is realized by an integrated circuit (LSI or IC), an external storage unit, communication with a cloud server via the Internet, broadcasting, or the like.

[0063] Next, point cloud data will be described. Fig. 2 is a diagram showing the configuration of point cloud data. Fig. 3 is a diagram showing an example of the configuration of a data file in which information about point cloud data is written.

[0064] Point cloud data contains data on multiple points. The data on each point includes location information (three-dimensional coordinates) and attribute information for that location information. A collection of multiple points is called a point cloud. For example, a point cloud can represent the three-dimensional shape of an object.

[0065] Position information such as three-dimensional coordinates is sometimes called geometry. Data for each point may also include attribute information of multiple attribute types. Attribute types include, for example, color or reflectance.

[0066] One piece of attribute information may be associated with one piece of location information, or multiple pieces of attribute information with different attribute types may be associated with one piece of location information, or multiple pieces of attribute information of the same attribute type may be associated with one piece of location information.

[0067] The configuration example of the data file shown in FIG. 3 is an example in which there is a one-to-one correspondence between position information and attribute information, and shows the position information and attribute information of N points that make up the point cloud data.

[0068] The position information is, for example, information on the three axes x, y, and z. The attribute information is, for example, RGB color information. A typical data file is a ply file.

[0069] Next, the types of point cloud data will be explained. Fig. 4 is a diagram showing the types of point cloud data. As shown in Fig. 4, point cloud data includes static objects and dynamic objects.

[0070] A static object is 3D point cloud data at any time (a certain time). A dynamic object is 3D point cloud data that changes over time. Hereinafter, 3D point cloud data at a certain time will be referred to as a PCC frame, or simply a frame.

[0071] The object may be a point cloud with a certain area restriction, such as ordinary video data, or a large-scale point cloud with no area restriction, such as map information.

[0072] Furthermore, there may be point cloud data of various densities, such as sparse point cloud data and dense point cloud data.

[0073] Each processing unit will be described in detail below. Sensor information is acquired by various methods, such as a distance sensor such as a LIDAR or a range finder, a stereo camera, or a combination of multiple monocular cameras. The point cloud data generation unit 4618 generates point cloud data based on the sensor information acquired by the sensor information acquisition unit 4617. The point cloud data generation unit 4618 generates position information as point cloud data, and adds attribute information for the position information to the position information.

[0074] The point cloud data generation unit 4618 may process the point cloud data when generating position information or adding attribute information. For example, the point cloud data generation unit 4618 may reduce the amount of data by deleting point clouds with overlapping positions. In addition, the point cloud data generation unit 4618 may convert (position shift, rotation, normalization, etc.) the position information or render the attribute information.

[0075] In FIG. 1, the point cloud data generation system 4611 is included in the three-dimensional data encoding system 4601, but it may be provided independently outside the three-dimensional data encoding system 4601.

[0076] The encoding unit 4613 generates encoded data by encoding the point cloud data based on a predetermined encoding method. There are two main types of encoding methods: the first is an encoding method that uses position information, and this encoding method will be referred to as the first encoding method hereinafter; and the second is an encoding method that uses a video codec, and this encoding method will be referred to as the second encoding method hereinafter.

[0077] The decoding unit 4624 decodes the coded data based on a predetermined coding method to decode the point group data.

[0078] The multiplexing unit 4614 generates multiplexed data by multiplexing the encoded data using an existing multiplexing method. The generated multiplexed data is transmitted or stored. In addition to the PCC encoded data, the multiplexing unit 4614 multiplexes other media such as video, audio, subtitles, applications, and files, or reference time information. The multiplexing unit 4614 may also multiplex attribute information related to sensor information or point cloud data.

[0079] Multiplexing methods or file formats include ISOBMFF, MPEG-DASH, which is an ISOBMFF-based transmission method, MMT, MPEG-2 TS Systems, and RMP.

[0080] The demultiplexer 4623 extracts PCC encoded data, other media, time information, and the like from the multiplexed data.

[0081] The input / output unit 4615 transmits the multiplexed data using a method suited to the transmission medium or storage medium, such as broadcasting or communication. The input / output unit 4615 may communicate with other devices via the Internet, or may communicate with a storage unit such as a cloud server.

[0082] The communication protocol used may be http, ftp, TCP, UDP, etc. A PULL type communication method or a PUSH type communication method may be used.

[0083] Either wired or wireless transmission may be used. For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), coaxial cable, etc. are used. For wireless transmission, wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), millimeter waves, etc. are used.

[0084] As a broadcasting system, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 is used.

[0085] Fig. 5 is a diagram showing the configuration of a first encoding unit 4630, which is an example of the encoding unit 4613 that performs encoding using the first encoding method. Fig. 6 is a block diagram of the first encoding unit 4630. The first encoding unit 4630 generates encoded data (encoded stream) by encoding point cloud data using the first encoding method. This first encoding unit 4630 includes a position information encoding unit 4631, an attribute information encoding unit 4632, an additional information encoding unit 4633, and a multiplexing unit 4634.

[0086] The first encoding unit 4630 is characterized in that it performs encoding taking into consideration a three-dimensional structure. The first encoding unit 4630 is also characterized in that the attribute information encoding unit 4632 performs encoding using information obtained from the position information encoding unit 4631. The first encoding method is also called GPCC (Geometry based PCC).

[0087] The point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData). The position information is input to a position information encoder 4631, the attribute information is input to an attribute information encoder 4632, and the additional information is input to an additional information encoder 4633.

[0088] The position information encoding unit 4631 encodes the position information to generate encoded position information (Compressed Geometry), which is encoded data. For example, the position information encoding unit 4631 encodes the position information using an N-ary tree structure such as an octree. Specifically, in an octree, the target space is divided into eight nodes (subspaces), and 8-bit information (occupancy code) indicating whether or not a point cloud is included in each node is generated. Furthermore, the node including the point cloud is further divided into eight nodes, and 8-bit information indicating whether or not a point cloud is included in each of the eight nodes is generated. This process is repeated until the number of point clouds included in a predetermined layer or node falls below a threshold.

[0089] The attribute information encoding unit 4632 generates encoded attribute information (Compressed Attribute) that is encoded data by encoding using the configuration information generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 determines a reference point (reference node) to be referenced when encoding a target point (target node) to be processed, based on the octree structure generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 references a peripheral node or adjacent node whose parent node in the octree is the same as that of the target node. Note that the method of determining the reference relationship is not limited to this.

[0090] Furthermore, the encoding process of the attribute information may include at least one of a quantization process, a prediction process, and an arithmetic coding process. In this case, the reference means using a reference node to calculate a predicted value of the attribute information, or using the state of the reference node (e.g., occupancy information indicating whether the reference node includes a point group) to determine an encoding parameter. For example, the encoding parameter is a quantization parameter in a quantization process, or a context in an arithmetic coding process.

[0091] The additional information encoding unit 4633 generates encoded additional information (Compressed MetaData) that is encoded data by encoding compressible data from the additional information.

[0092] The multiplexing unit 4634 multiplexes the encoding position information, the encoding attribute information, the encoding additional information, and other additional information to generate a compressed stream, which is encoded data. The generated compressed stream is output to a processing unit in a system layer (not shown).

[0093] Next, a first decoding unit 4640, which is an example of the decoding unit 4624 that performs decoding using the first encoding method, will be described. FIG. 7 is a diagram showing the configuration of the first decoding unit 4640. FIG. 8 is a block diagram of the first decoding unit 4640. The first decoding unit 4640 generates point cloud data by decoding, using the first encoding method, coded data (coded stream) coded using the first coding method. The first decoding unit 4640 includes a demultiplexing unit 4641, a position information decoding unit 4642, an attribute information decoding unit 4643, and an additional information decoding unit 4644.

[0094] A coded stream (compressed stream) that is coded data is input to the first decoding unit 4640 from a processing unit in a system layer (not shown).

[0095] The demultiplexer 4641 separates the encoded position information (Compressed Geometry), the encoded attribute information (Compressed Attribute), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0096] The position information decoding unit 4642 generates position information by decoding the encoded position information. For example, the position information decoding unit 4642 restores the position information of a point group represented by three-dimensional coordinates from the encoded position information represented by an N-ary tree structure such as an octree.

[0097] The attribute information decoding unit 4643 decodes the encoded attribute information based on the configuration information generated by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 determines a reference point (reference node) to be referenced in decoding the target point (target node) to be processed based on the octree structure obtained by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 references a peripheral node or adjacent node whose parent node in the octree is the same as that of the target node. Note that the method of determining the reference relationship is not limited to this.

[0098] Furthermore, the attribute information decoding process may include at least one of an inverse quantization process, a prediction process, and an arithmetic decoding process. In this case, the reference means using a reference node to calculate a predicted value of the attribute information, or using the state of the reference node (e.g., occupancy information indicating whether the reference node includes a point group) to determine a decoding parameter. For example, the decoding parameter is a quantization parameter in an inverse quantization process, or a context in an arithmetic decoding process.

[0099] The additional information decoding unit 4644 generates additional information by decoding the encoded additional information. The first decoding unit 4640 uses the additional information necessary for decoding the position information and attribute information during decoding, and outputs the additional information necessary for the application to the outside.

[0100] Next, an example of the configuration of the position information encoding unit will be described. Fig. 9 is a block diagram of position information encoding unit 2700 according to this embodiment. Position information encoding unit 2700 includes an octree generation unit 2701, a geometric information calculation unit 2702, a coding table selection unit 2703, and an entropy encoding unit 2704.

[0101] The octree generation unit 2701 generates, for example, an occupancy tree from the input position information, and generates an occupancy code for each node in the occupancy tree. The geometric information calculation unit 2702 acquires information indicating whether an adjacent node of the target node is an occupied node. For example, the geometric information calculation unit 2702 calculates the occupancy information of the adjacent node (information indicating whether the adjacent node is an occupied node) from the occupancy code of the parent node to which the target node belongs. The geometric information calculation unit 2702 may also store encoded nodes in a list and search for adjacent nodes from the list. The geometric information calculation unit 2702 may also switch adjacent nodes depending on the position of the target node within the parent node.

[0102] The coding table selection unit 2703 selects a coding table to be used for entropy coding of the target node using the occupancy information of the adjacent nodes calculated by the geometric information calculation unit 2702. For example, the coding table selection unit 2703 may generate a bit string using the occupancy information of the adjacent nodes, and select a coding table for an index number generated from the bit string.

[0103] The entropy coding unit 2704 generates the coding position information and metadata by entropy coding the occupancy code of the target node using the coding table of the selected index number. The entropy coding unit 2704 may add information indicating the selected coding table to the coding position information.

[0104] The octree representation and the scanning order of position information will be explained below. Position information (position data) is converted (octreeized) into an octree structure and then encoded. The octree structure consists of nodes and leaves. Each node has eight nodes or leaves, and each leaf has voxel (VXL) information. Fig. 10 is a diagram showing an example of the structure of position information containing multiple voxels. Fig. 11 is a diagram showing an example of the position information shown in Fig. 10 converted into an octree structure. Here, among the leaves shown in Fig. 11, leaves 1, 2, and 3 represent voxels VXL1, VXL2, and VXL3 shown in Fig. 10, respectively, and represent a VXL containing a point cloud (hereinafter referred to as effective VXL).

[0105] Specifically, node 1 corresponds to the entire space including the position information in Figure 10. The entire space corresponding to node 1 is divided into eight nodes, and of the eight nodes, the node containing a valid VXL is further divided into eight nodes or leaves, and this process is repeated for each level of the tree structure. Here, each node corresponds to a subspace, and has information (occupancy code) as node information indicating at what position the next node or leaf will be located after division. In addition, the block at the bottom level is set as a leaf, and the number of point clouds contained in the leaf, etc., is held as leaf information.

[0106] Next, an example of the configuration of the position information decoding unit will be described. Fig. 12 is a block diagram of the position information decoding unit 2710 according to this embodiment. The position information decoding unit 2710 includes an octree generation unit 2711, a geometric information calculation unit 2712, a coding table selection unit 2713, and an entropy decoding unit 2714.

[0107] The octree generation unit 2711 generates an octree of a certain space (node) using header information or metadata of a bitstream. For example, the octree generation unit 2711 generates a large space (root node) using the sizes of the x-axis, y-axis, and z-axis directions of a certain space added to the header information, and generates an octree by dividing the space into two in the x-axis, y-axis, and z-axis directions to generate eight small spaces A (nodes A0 to A7). In addition, nodes A0 to A7 are set in order as target nodes.

[0108] The geometric information calculation unit 2712 acquires occupancy information indicating whether an adjacent node of the target node is an occupied node. For example, the geometric information calculation unit 2712 calculates the occupancy information of the adjacent node from the occupancy code of the parent node to which the target node belongs. The geometric information calculation unit 2712 may also store decoded nodes in a list and search for adjacent nodes from the list. The geometric information calculation unit 2712 may also switch adjacent nodes depending on the position of the target node within the parent node.

[0109] The coding table selection unit 2713 selects a coding table (decoding table) to be used for entropy decoding of the target node using the occupancy information of the adjacent nodes calculated by the geometric information calculation unit 2712. For example, the coding table selection unit 2713 may generate a bit string using the occupancy information of the adjacent nodes, and select a coding table for an index number generated from the bit string.

[0110] The entropy decoding unit 2714 generates position information by entropy decoding the occupancy code of the target node using the selected coding table. Note that the entropy decoding unit 2714 may obtain information on the selected coding table by decoding it from the bitstream, and entropy decode the occupancy code of the target node using the coding table indicated by the information.

[0111] The configurations of the attribute information encoding unit and the attribute information decoding unit will be described below. Fig. 13 is a block diagram showing an example configuration of the attribute information encoding unit A100. The attribute information encoding unit may include multiple encoding units that execute different encoding methods. For example, the attribute information encoding unit may switch between the following two methods depending on the use case:

[0112] The attribute information encoding unit A100 includes an LoD attribute information encoding unit A101 and a conversion attribute information encoding unit A102. The LoD attribute information encoding unit A101 classifies each 3D point into multiple layers using position information of the 3D points, predicts attribute information of the 3D points belonging to each layer, and encodes the prediction residual. Here, each classified layer is called an LoD (Level of Detail).

[0113] The transformed attribute information encoding unit A102 encodes the attribute information using RAHT (Region Adaptive Hierarchical Transform). Specifically, the transformed attribute information encoding unit A102 applies RAHT or Haar transform to each piece of attribute information based on the position information of the three-dimensional point to generate high-frequency components and low-frequency components for each layer, and encodes these values ​​using quantization, entropy coding, etc.

[0114] 14 is a block diagram showing an example configuration of the attribute information decoding unit A110. The attribute information decoding unit may include multiple decoding units that execute different decoding methods. For example, the attribute information decoding unit may switch between the following two methods based on information included in the header or metadata for decoding:

[0115] The attribute information decoding unit A110 includes an LoD attribute information decoding unit A111 and a converted attribute information decoding unit A112. The LoD attribute information decoding unit A111 classifies each 3D point into multiple layers using the position information of the 3D points, and decodes the attribute values ​​while predicting the attribute information of the 3D points belonging to each layer.

[0116] The transformed attribute information decoding unit A112 decodes the attribute information using RAHT (Region Adaptive Hierarchical Transform). Specifically, the transformed attribute information decoding unit A112 decodes the attribute values ​​by applying inverse RAHT or inverse Haar transform to the high-frequency components and low-frequency components of each attribute value based on the position information of the three-dimensional point.

[0117] FIG. 15 is a block diagram showing the configuration of an attribute information encoding unit 3140, which is an example of the LoD attribute information encoding unit A101.

[0118] The attribute information encoding unit 3140 includes an LoD generation unit 3141, a surrounding search unit 3142, a prediction unit 3143, a prediction residual calculation unit 3144, a quantization unit 3145, an arithmetic encoding unit 3146, an inverse quantization unit 3147, a decoded value generation unit 3148, and a memory 3149.

[0119] The LoD generation unit 3141 generates LoD using the position information of the three-dimensional points.

[0120] The surrounding search unit 3142 searches for nearby 3D points adjacent to each 3D point using the LoD generation result by the LoD generation unit 3141 and distance information indicating the distance between each 3D point.

[0121] The prediction unit 3143 generates a predicted value of the attribute information of the target 3D point to be coded.

[0122] The prediction residual calculation unit 3144 calculates (generates) a prediction residual of the predicted value of the attribute information generated by the prediction unit 3143.

[0123] The quantization unit 3145 quantizes the prediction residual of the attribute information calculated by the prediction residual calculation unit 3144 .

[0124] The arithmetic coding unit 3146 arithmetically codes the prediction residuals after being quantized by the quantization unit 3145. The arithmetic coding unit 3146 outputs a bitstream including the arithmetically coded prediction residuals to, for example, a three-dimensional data decoding device.

[0125] Note that the prediction residual may be binarized by, for example, the quantization unit 3145 before being arithmetically coded by the arithmetic coding unit 3146.

[0126] Furthermore, for example, the arithmetic coding unit 3146 may initialize a coding table used for arithmetic coding before arithmetic coding. The arithmetic coding unit 3146 may initialize a coding table used for arithmetic coding for each layer. Furthermore, the arithmetic coding unit 3146 may output information indicating the position of the layer for which the coding table has been initialized, by including it in the bitstream.

[0127] The inverse quantization unit 3147 inverse quantizes the prediction residuals quantized by the quantization unit 3145 .

[0128] The decoded value generation unit 3148 generates a decoded value by adding the predicted value of the attribute information generated by the prediction unit 3143 and the prediction residual after inverse quantization by the inverse quantization unit 3147.

[0129] The memory 3149 is a memory that stores the decoded values ​​of the attribute information of each 3D point decoded by the decoded value generation unit 3148. For example, when generating a predicted value of a 3D point that has not yet been encoded, the prediction unit 3143 generates the predicted value by using the decoded values ​​of the attribute information of each 3D point stored in the memory 3149.

[0130] 16 is a block diagram of an attribute information encoding unit 6600, which is an example of the transformed attribute information encoding unit A102. The attribute information encoding unit 6600 includes a sorting unit 6601, a Haar transform unit 6602, a quantization unit 6603, an inverse quantization unit 6604, an inverse Haar transform unit 6605, a memory 6606, and an arithmetic encoding unit 6607.

[0131] The sorting unit 6601 generates a Morton code using the position information of the 3D points and sorts the multiple 3D points in Morton code order. The Haar transform unit 6602 generates coding coefficients by applying a Haar transform to the attribute information. The quantization unit 6603 quantizes the coding coefficients of the attribute information.

[0132] The inverse quantization unit 6604 inversely quantizes the quantized coding coefficients. The inverse Haar transform unit 6605 applies an inverse Haar transform to the coding coefficients. The memory 6606 stores values ​​of attribute information of multiple decoded 3D points. For example, the attribute information of the decoded 3D points stored in the memory 6606 may be used for predicting uncoded 3D points.

[0133] The arithmetic coding unit 6607 calculates ZeroCnt from the quantized coding coefficients and arithmetically codes the ZeroCnt. The arithmetic coding unit 6607 also arithmetically codes the quantized non-zero coding coefficients. The arithmetic coding unit 6607 may binarize the coding coefficients before arithmetic coding. The arithmetic coding unit 6607 may also generate and code various header information.

[0134] FIG. 17 is a block diagram showing the configuration of an attribute information decoding unit 3150, which is an example of the LoD attribute information decoding unit A111.

[0135] The attribute information decoding unit 3150 includes an LoD generation unit 3151 , a surrounding search unit 3152 , a prediction unit 3153 , an arithmetic decoding unit 3154 , an inverse quantization unit 3155 , a decoded value generation unit 3156 , and a memory 3157 .

[0136] The LoD generation unit 3151 generates LoD using the position information of the 3D points decoded by the position information decoding unit (not shown in FIG. 17).

[0137] The surrounding search unit 3152 searches for nearby 3D points adjacent to each 3D point using the LoD generation result by the LoD generation unit 3151 and distance information indicating the distance between each 3D point.

[0138] The prediction unit 3153 generates a predicted value of the attribute information of the target 3D point to be decoded.

[0139] The arithmetic decoding unit 3154 arithmetically decodes prediction residuals in the bitstream acquired from the attribute information coding unit 3140 shown in FIG. 15. The arithmetic decoding unit 3154 may initialize a decoding table used for arithmetic decoding. The arithmetic decoding unit 3154 initializes a decoding table used for arithmetic decoding for a layer on which the arithmetic coding unit 3146 shown in FIG. 15 has performed coding processing. The arithmetic decoding unit 3154 may initialize a decoding table used for arithmetic decoding for each layer. The arithmetic decoding unit 3154 may initialize the decoding table based on information included in the bitstream that indicates the position of the layer for which the coding table has been initialized.

[0140] The inverse quantization unit 3155 inverse quantizes the prediction residuals arithmetically decoded by the arithmetic decoding unit 3154 .

[0141] The decoded value generation unit 3156 generates a decoded value by adding the predicted value generated by the prediction unit 3153 and the prediction residual after inverse quantization by the inverse quantization unit 3155. The decoded value generation unit 3156 outputs the decoded attribute information data to another device.

[0142] The memory 3157 is a memory that stores the decoded values ​​of the attribute information of each 3D point decoded by the decoded value generation unit 3156. For example, when generating a predicted value of a 3D point that has not yet been decoded, the prediction unit 3153 generates the predicted value by using the decoded value of the attribute information of each 3D point stored in the memory 3157.

[0143] 18 is a block diagram of an attribute information decoding unit 6610, which is an example of the transformed attribute information decoding unit A112. The attribute information decoding unit 6610 includes an arithmetic decoding unit 6611, an inverse quantization unit 6612, an inverse Haar transform unit 6613, and a memory 6614.

[0144] The arithmetic decoding unit 6611 arithmetically decodes the ZeroCnt and coding coefficients included in the bitstream. Note that the arithmetic decoding unit 6611 may also decode various types of header information.

[0145] The inverse quantization unit 6612 inverse quantizes the arithmetically decoded coding coefficients. The inverse Haar transform unit 6613 applies inverse Haar transform to the inversely quantized coding coefficients. The memory 6614 stores values ​​of attribute information of multiple decoded 3D points. For example, the attribute information of the decoded 3D points stored in the memory 6614 may be used to predict undecoded 3D points.

[0146] Next, a description will be given of second encoding unit 4650, which is an example of encoding unit 4613 that performs encoding using the second encoding method. Fig. 19 is a diagram showing the configuration of second encoding unit 4650. Fig. 20 is a block diagram of second encoding unit 4650.

[0147] The second encoding unit 4650 generates encoded data (encoded stream) by encoding the point cloud data using a second encoding method. The second encoding unit 4650 includes an additional information generation unit 4651, a position image generation unit 4652, an attribute image generation unit 4653, a video encoding unit 4654, an additional information encoding unit 4655, and a multiplexing unit 4656.

[0148] The second encoding unit 4650 has a feature of generating a position image and an attribute image by projecting a three-dimensional structure onto a two-dimensional image, and encoding the generated position image and attribute image using an existing video encoding method. The second encoding method is also called VPCC (Video based PCC).

[0149] The point cloud data is PCC point cloud data such as a PLY file or PCC point cloud data generated from sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData).

[0150] The additional information generating unit 4651 generates map information of a plurality of two-dimensional images by projecting a three-dimensional structure onto the two-dimensional images.

[0151] The position image generation unit 4652 generates a position image (Geometry Image) based on the position information and the map information generated by the additional information generation unit 4651. This position image is, for example, a distance image in which distance (Depth) is indicated as a pixel value. Note that this distance image may be an image in which multiple point clouds are viewed from a single viewpoint (an image in which multiple point clouds are projected onto a single two-dimensional plane), or may be multiple images in which multiple point clouds are viewed from multiple viewpoints, or may be a single image in which these multiple images are integrated.

[0152] The attribute image generation unit 4653 generates an attribute image based on the attribute information and the map information generated by the additional information generation unit 4651. This attribute image is, for example, an image in which attribute information (for example, color (RGB)) is represented as pixel values. Note that this image may be an image in which multiple point clouds are viewed from one viewpoint (an image in which multiple point clouds are projected onto one two-dimensional plane), or multiple images in which multiple point clouds are viewed from multiple viewpoints, or a single image in which these multiple images are integrated.

[0153] The video encoding unit 4654 generates an encoded position image (Compressed Geometry Image) and an encoded attribute image (Compressed Attribute Image), which are encoded data, by encoding the position image and the attribute image using a video encoding method. Note that any known encoding method may be used as the video encoding method. For example, the video encoding method is AVC, HEVC, or the like.

[0154] The additional information encoding unit 4655 generates encoded additional information (Compressed MetaData) by encoding the additional information, map information, and the like included in the point cloud data.

[0155] The multiplexing unit 4656 multiplexes the encoding position image, the encoding attribute image, the encoding additional information, and other additional information to generate an encoded stream (Compressed Stream) that is encoded data. The generated encoded stream is output to a processing unit in a system layer (not shown).

[0156] Next, a second decoding unit 4660, which is an example of the decoding unit 4624 that performs decoding using the second encoding method, will be described. FIG. 21 is a diagram showing the configuration of the second decoding unit 4660. FIG. 22 is a block diagram of the second decoding unit 4660. The second decoding unit 4660 generates point cloud data by decoding, using the second encoding method, coded data (coded stream) that has been coded using the second coding method. The second decoding unit 4660 includes a demultiplexing unit 4661, a video decoding unit 4662, an additional information decoding unit 4663, a position information generation unit 4664, and an attribute information generation unit 4665.

[0157] A coded stream (compressed stream) that is coded data is input to the second decoding unit 4660 from a processing unit in a system layer (not shown).

[0158] The demultiplexer 4661 separates the encoded position image (Compressed Geometry Image), the encoded attribute image (Compressed Attribute Image), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0159] The video decoding unit 4662 generates a position image and an attribute image by decoding the encoded position image and the encoded attribute image using a video encoding method. Note that any known encoding method may be used as the video encoding method. For example, the video encoding method is AVC or HEVC.

[0160] The additional information decoding unit 4663 decodes the encoded additional information to generate additional information including map information and the like.

[0161] The position information generating unit 4664 generates position information using the position image and map information. The attribute information generating unit 4665 generates attribute information using the attribute image and map information.

[0162] The second decoding unit 4660 uses the additional information necessary for decoding during decoding, and outputs the additional information necessary for the application to the outside.

[0163] The following describes the problems with the PCC encoding method. Fig. 23 is a diagram showing a protocol stack related to PCC encoded data. Fig. 23 shows an example in which other media data such as video (e.g., HEVC) or audio is multiplexed with PCC encoded data and transmitted or stored.

[0164] Multiplexing methods and file formats have the function of multiplexing various coded data and transmitting or storing them. To transmit or store coded data, the coded data must be converted into the format of the multiplexing method. For example, HEVC specifies a technology that stores coded data in a data structure called a NAL unit and stores the NAL unit in ISOBMFF.

[0165] On the other hand, currently, a first encoding method (Codec1) and a second encoding method (Codec2) are being considered as methods for encoding point cloud data, but the structure of the encoded data and the method for storing the encoded data in a system format have not been defined, which poses the problem that, as it stands, it is not possible to perform MUX processing (multiplexing) in the encoding unit, transmission, or storage.

[0166] In the following description, unless a specific encoding method is specified, it refers to either the first encoding method or the second encoding method.

[0167] (Embodiment 2) In this embodiment, the types of coded data (position information (Geometry), attribute information (Attribute), additional information (Metadata)) generated by the first coding unit 4630 or the second coding unit 4650 described above, a method for generating the additional information (Metadata), and multiplexing processing in the multiplexing unit will be described. Note that the additional information (Metadata) may also be referred to as a parameter set or control information.

[0168] In this embodiment, the dynamic object (three-dimensional point cloud data that changes over time) described in Figure 4 will be used as an example, but a similar method may also be used in the case of a static object (three-dimensional point cloud data at any time).

[0169] 24 is a diagram showing the configurations of an encoding unit 4801 and a multiplexing unit 4802 included in the three-dimensional data encoding device according to this embodiment. The encoding unit 4801 corresponds to, for example, the first encoding unit 4630 or the second encoding unit 4650 described above. The multiplexing unit 4802 corresponds to the multiplexing unit 4634 or 4656 described above.

[0170] The encoding unit 4801 encodes point cloud data of multiple PCC (Point Cloud Compression) frames, and generates encoded data (Multiple Compressed Data) of multiple pieces of position information, attribute information, and additional information.

[0171] The multiplexing unit 4802 converts data of multiple data types (position information, attribute information, and additional information) into NAL units, thereby converting the data into a data structure that takes into account data access in the decoding device.

[0172] 25 is a diagram showing an example of the structure of coded data generated by coding unit 4801. Arrows in the diagram indicate dependencies related to the decoding of coded data, with the source of the arrow depending on the data at the end of the arrow. In other words, the decoding device decodes the data at the end of the arrow and uses the decoded data to decode the data at the end of the arrow. In other words, dependency means that the data on which the dependency is based is referenced (used) in the processing (encoding, decoding, etc.) of the data on which the dependency is based.

[0173] First, the process of generating encoded data of position information will be described. The encoding unit 4801 generates encoded position data (compressed geometry data) for each frame by encoding the position information of each frame. The encoded position data is represented by G(i), where i indicates the frame number, the time of the frame, etc.

[0174] The encoding unit 4801 also generates a position parameter set (GPS(i)) corresponding to each frame. The position parameter set includes parameters that can be used to decode the encoded position data. The encoded position data for each frame depends on the corresponding position parameter set.

[0175] Furthermore, encoded position data consisting of multiple frames is defined as a position sequence (Geometry Sequence). The encoding unit 4801 generates a position sequence parameter set (Geometry Sequence PS: also referred to as position SPS) that stores parameters commonly used in decoding processes for multiple frames in the position sequence. The position sequence depends on the position SPS.

[0176] Next, the process of generating coded data of attribute information will be described. The coding unit 4801 generates coded attribute data (Compressed Attribute Data) for each frame by coding the attribute information of each frame. The coded attribute data is represented by A(i). FIG. 25 shows an example in which attribute X and attribute Y exist, and the coded attribute data of attribute X is represented by AX(i) and the coded attribute data of attribute Y is represented by AY(i).

[0177] The encoding unit 4801 also generates an attribute parameter set (APS(i)) corresponding to each frame. The attribute parameter set for attribute X is represented as AXPS(i), and the attribute parameter set for attribute Y is represented as AYPS(i). The attribute parameter set includes parameters that can be used to decode encoded attribute information. The encoded attribute data depends on the corresponding attribute parameter set.

[0178] Furthermore, encoded attribute data consisting of multiple frames is defined as an attribute sequence. The encoding unit 4801 generates an attribute sequence parameter set (Attribute Sequence PS: also referred to as attribute SPS) that stores parameters commonly used in decoding processes for multiple frames in the attribute sequence. The attribute sequence depends on the attribute SPS.

[0179] Furthermore, in the first encoding method, the encoded attribute data depends on the encoded position data.

[0180] 25 shows an example in which two types of attribute information (attribute X and attribute Y) exist. When there are two types of attribute information, for example, two encoding units generate respective data and metadata. Also, for example, an attribute sequence is defined for each type of attribute information, and an attribute SPS is generated for each type of attribute information.

[0181] 25 shows an example in which there is one type of position information and two types of attribute information, but this is not limiting, and there may be one type of attribute information, or three or more types. In this case, encoded data can be generated using a similar method. Furthermore, in the case of point cloud data that does not have attribute information, the attribute information may not be necessary. In this case, the encoding unit 4801 does not need to generate a parameter set related to the attribute information.

[0182] Next, the process of generating additional information (metadata) will be described. The encoding unit 4801 generates a PCC Stream PS (also referred to as stream PS), which is a parameter set for the entire PCC stream. The encoding unit 4801 stores, in the stream PS, parameters that can be used in common for decoding processes for one or more position sequences and one or more attribute sequences. For example, the stream PS includes identification information indicating the codec for the point cloud data, information indicating the algorithm used for encoding, and the like. The position sequence and attribute sequence depend on the stream PS.

[0183] Next, the access unit and GOF will be explained. In this embodiment, the concepts of the access unit (AU) and GOF (Group of Frame) are newly introduced.

[0184] An access unit is a basic unit for accessing data during decoding, and is composed of one or more pieces of data and one or more pieces of metadata. For example, an access unit is composed of position information at the same time and one or more pieces of attribute information. A GOF is a random access unit and is composed of one or more access units.

[0185] The encoding unit 4801 generates an access unit header (AU Header) as identification information indicating the beginning of an access unit. The encoding unit 4801 stores parameters related to the access unit in the access unit header. For example, the access unit header includes the configuration or information of the coded data included in the access unit. The access unit header also includes parameters commonly used for the data included in the access unit, such as parameters related to decoding of the coded data.

[0186] Instead of an access unit header, the encoding unit 4801 may generate an access unit delimiter that does not include parameters related to the access unit. This access unit delimiter is used as identification information indicating the start of the access unit. The decoding device identifies the start of the access unit by detecting the access unit header or the access unit delimiter.

[0187] Next, the generation of identification information for the start of a GOF will be described. The encoding unit 4801 generates a GOF header as identification information that indicates the start of a GOF. The encoding unit 4801 stores parameters related to the GOF in the GOF header. For example, the GOF header includes the configuration or information of the coded data included in the GOF. The GOF header also includes parameters commonly used for the data included in the GOF, such as parameters related to the decoding of the coded data.

[0188] Instead of a GOF header, the encoding unit 4801 may generate a GOF delimiter that does not include parameters related to the GOF. This GOF delimiter is used as identification information that indicates the beginning of the GOF. The decoding device identifies the beginning of the GOF by detecting the GOF header or the GOF delimiter.

[0189] In PCC encoded data, for example, an access unit is defined as a PCC frame unit, and a decoding device accesses a PCC frame based on identification information at the beginning of the access unit.

[0190] Also, for example, GOF is defined as one random access unit. A decoding device accesses the random access unit based on the identification information at the beginning of the GOF. For example, if PCC frames are not dependent on each other and can be decoded independently, the PCC frames may be defined as the random access unit.

[0191] It should be noted that two or more PCC frames may be allocated to one access unit, and multiple random access units may be allocated to one GOF.

[0192] The encoding unit 4801 may also define and generate parameter sets or metadata other than those described above. For example, the encoding unit 4801 may generate SEI (Supplemental Enhancement Information) that stores parameters (optional parameters) that may not necessarily be used during decoding.

[0193] Next, the structure of coded data and the method of storing coded data in NAL units will be described.

[0194] For example, a data format is defined for each type of coded data. Figure 26 shows examples of coded data and NAL units.

[0195] For example, as shown in Fig. 26, the coded data includes a header and a payload. The coded data may include length information indicating the length (amount of data) of the coded data, the header, or the payload. The coded data may not include a header.

[0196] The header includes, for example, identification information for identifying the data, such as the data type or frame number.

[0197] The header includes, for example, identification information indicating a reference relationship. This identification information is stored in the header when, for example, there is a dependency relationship between data, and is information for referencing the reference destination from the reference source. For example, the header of the reference destination includes identification information for identifying the data. The header of the reference source includes identification information indicating the reference destination.

[0198] If the reference destination or the reference source can be identified or derived from other information, the identification information for specifying the data or the identification information indicating the reference relationship may be omitted.

[0199] The multiplexing unit 4802 stores the coded data in the payload of the NAL unit. The NAL unit header includes pcc_nal_unit_type, which is information identifying the coded data. Figure 27 shows an example of the semantics of pcc_nal_unit_type.

[0200] As shown in Fig. 27, when pcc_codec_type is codec 1 (Codec1: first encoding method), values ​​0 to 10 of pcc_nal_unit_type are assigned to the encoded position data (Geometry), encoded attribute X data (AttributeX), encoded attribute Y data (AttributeY), position PS (Geom.PS), attribute XPS (AttrX.PS), attribute YPS (AttrX.PS), position SPS (Geometry Sequence PS), attribute XSPS (AttributeX Sequence PS), attribute YSPS (AttributeY Sequence PS), AU header (AU Header), and GOF header (GOF Header) in codec 1. Values ​​11 and above are assigned as spares for codec 1.

[0201] When pcc_codec_type is Codec2 (Codec2: second encoding method), values ​​0 to 2 of pcc_nal_unit_type are assigned to codec data A (DataA), metadata A (MetaDataA), and metadata B (MetaDataB). Values ​​3 and above are assigned as spares for Codec2.

[0202] Next, the data transmission order will be explained. The following explains the restrictions on the transmission order of NAL units.

[0203] The multiplexing unit 4802 collectively transmits NAL units in units of GOF or AU. The multiplexing unit 4802 places a GOF header at the beginning of a GOF, and places an AU header at the beginning of an AU.

[0204] The multiplexing unit 4802 may allocate a sequence parameter set (SPS) for each AU so that the decoding device can decode from the next AU even if data is lost due to packet loss or the like.

[0205] If the coded data has a dependency relationship related to decoding, the decoding device decodes the referenced data first, and then decodes the referenced data. In order to enable the decoding device to decode the data in the order in which it was received without rearranging the data, the multiplexing unit 4802 sends the referenced data first.

[0206] 28 is a diagram showing examples of the transmission order of NAL units, and shows three examples: position information priority, parameter priority, and data integration.

[0207] The location information priority transmission order is an example in which information related to location information and information related to attribute information are transmitted together. In this transmission order, the transmission of information related to location information is completed earlier than the transmission of information related to attribute information.

[0208] For example, by using this transmission order, a decoding device that does not decode attribute information may be able to set a time during which it does not process the attribute information by ignoring the decoding of the attribute information. Also, for example, in the case of a decoding device that wants to decode position information quickly, it may be able to decode the position information more quickly by obtaining the encoded data of the position information early.

[0209] In FIG. 28, the attributes XSPS and YSPS are combined and written as the attribute SPS, but the attributes XSPS and YSPS may be arranged separately.

[0210] In the parameter set priority sending order, the parameter sets are sent first and the data is sent later.

[0211] As long as the NAL unit transmission order constraints are met as described above, the multiplexing unit 4802 may transmit NAL units in any order. For example, order identification information may be defined, and the multiplexing unit 4802 may have the function of transmitting NAL units in multiple order patterns. For example, the NAL unit order identification information may be stored in the stream PS.

[0212] The three-dimensional data decoding device may perform decoding based on the order identification information. The three-dimensional data decoding device may instruct the three-dimensional data encoding device on a desired transmission order, and the three-dimensional data encoding device (multiplexing unit 4802) may control the transmission order in accordance with the instructed transmission order.

[0213] The multiplexing unit 4802 may generate coded data that merges multiple functions, as long as the data transmission order is within the constraints of the transmission order, such as the transmission order of the integrated data. For example, as shown in Fig. 28, the GOF header and the AU header may be integrated, or the AXPS and the AYPS may be integrated. In this case, an identifier indicating that the data has multiple functions is defined in pcc_nal_unit_type.

[0214] A modified example of this embodiment will be described below. PS has levels, such as a frame-level PS, a sequence-level PS, and a PCC sequence-level PS, and if the PCC sequence level is the higher level and the frame level is the lower level, the following method may be used to store parameters.

[0215] The default PS value is indicated in the higher PS. Also, if the value of the lower PS differs from the value of the higher PS, the PS value is indicated in the lower PS. Alternatively, the PS value is not written in the higher PS, but written in the lower PS. Alternatively, information on whether the PS value is to be written in the lower PS, the higher PS, or both is written in either the lower PS or the higher PS, or both. Alternatively, the lower PS may be merged with the higher PS. Alternatively, if the lower PS and the higher PS overlap, the multiplexing unit 4802 may omit sending one of them.

[0216] The encoding unit 4801 or the multiplexing unit 4802 may divide the data into slices or tiles and transmit the divided data. The divided data includes information for identifying the divided data, and parameters used to decode the divided data are included in the parameter set. In this case, an identifier indicating that the data stores data or parameters related to tiles or slices is defined in pcc_nal_unit_type.

[0217] The processing related to the order identification information will be explained below. Fig. 29 is a flowchart of processing by the three-dimensional data encoding device (the encoding unit 4801 and the multiplexing unit 4802) related to the transmission order of NAL units.

[0218] First, the three-dimensional data encoding device determines the transmission order of NAL units (position information priority or parameter set priority) (S4801). For example, the three-dimensional data encoding device determines the transmission order based on a specification from a user or an external device (e.g., a three-dimensional data decoding device).

[0219] If the determined transmission order is positional information priority (positional information priority in S4802), the three-dimensional data encoding device sets the order identification information included in the stream PS to positional information priority (S4803). That is, in this case, the order identification information indicates that the NAL units are to be transmitted in an order that prioritizes positional information. The three-dimensional data encoding device then transmits the NAL units in positional information priority order (S4804).

[0220] On the other hand, if the determined transmission order is parameter set priority (parameter set priority in S4802), the three-dimensional data encoding device sets the order identification information included in the stream PS to parameter set priority (S4805). That is, in this case, the order identification information indicates that the NAL units will be transmitted in parameter set priority order. Then, the three-dimensional data encoding device transmits the NAL units in parameter set priority order (S4806).

[0221] 30 is a flowchart of processing by the 3D data decoding device related to the transmission order of NAL units. First, the 3D data decoding device analyzes the order identification information included in the stream PS (S4811).

[0222] If the transmission order indicated by the order identification information is position information priority (position information priority in S4812), the three-dimensional data decoding device decodes the NAL units assuming that the transmission order of the NAL units is position information priority (S4813).

[0223] On the other hand, if the transmission order indicated by the order identification information is parameter set priority (parameter set priority in S4812), the three-dimensional data decoding device decodes the NAL units assuming that the transmission order of the NAL units is parameter set priority (S4814).

[0224] For example, if the three-dimensional data decoding device does not decode attribute information, in step S4813, it may acquire NAL units related to position information without acquiring all NAL units, and decode the position information from the acquired NAL units.

[0225] Next, the processing related to the generation of AUs and GOFs will be described. Fig. 31 is a flowchart of processing by the 3D data encoding device (the multiplexing unit 4802) related to the generation of AUs and GOFs in multiplexing NAL units.

[0226] First, the three-dimensional data encoding device determines the type of encoded data (S4821). Specifically, the three-dimensional data encoding device determines whether the encoded data to be processed is the first data of an AU, the first data of a GOF, or other data.

[0227] If the coded data is data at the start of a GOF (GOF start in S4822), the three-dimensional data coding device places a GOF header and an AU header at the start of the coded data belonging to the GOF to generate a NAL unit (S4823).

[0228] If the coded data is data at the beginning of an AU (AU beginning in S4822), the three-dimensional data coding device places an AU header at the beginning of the coded data belonging to the AU and generates a NAL unit (S4824).

[0229] If the encoded data is not at the beginning of a GOF or an AU (other than the beginning of a GOF or an AU in S4822), the three-dimensional data encoding device places the encoded data after the AU header of the AU to which the encoded data belongs and generates a NAL unit (S4825).

[0230] Next, a process for accessing AUs and GOFs will be described. Fig. 32 is a flowchart of a process performed by a three-dimensional data decoding device for accessing AUs and GOFs during demultiplexing of NAL units.

[0231] First, the three-dimensional data decoding device determines the type of coded data included in the NAL unit by analyzing the nal_unit_type included in the NAL unit (S4831). Specifically, the three-dimensional data decoding device determines whether the coded data included in the NAL unit is data at the beginning of an AU, data at the beginning of a GOF, or other data.

[0232] If the encoded data contained in the NAL unit is the data at the start of the GOF (start of GOF in S4832), the three-dimensional data decoding device determines that the NAL unit is the start position of random access, accesses the NAL unit, and starts the decoding process (S4833).

[0233] On the other hand, if the coded data contained in the NAL unit is data at the beginning of an AU (AU beginning in S4832), the three-dimensional data decoding device determines that the NAL unit is the beginning of an AU, accesses the data contained in the NAL unit, and decodes the AU (S4834).

[0234] On the other hand, if the coded data included in the NAL unit is neither the start of a GOF nor the start of an AU (other than the start of a GOF or an AU in S4832), the three-dimensional data decoding device does not process the NAL unit.

[0235] (Embodiment 3) Fig. 33 is a block diagram showing the configuration of a three-dimensional data encoding device according to this embodiment. Note that in Fig. 33, an encoding unit that encodes position information, which is included in the three-dimensional data encoding device, is not shown.

[0236] The three-dimensional data encoding device 10600 includes a conversion unit 10610 and an encoding unit 10620.

[0237] The conversion unit 10610 performs conversion processing on the input attribute information before inputting it to the encoding unit 10620. The conversion processing is, for example, at least one of offsetting (offset processing) and scaling (scaling processing) described below.

[0238] The conversion unit 10610 has a scale unit 10611 and an offset unit 10612. It is sufficient that the conversion unit 10610 has at least one of the offset unit 10612 and the scale unit 10611. For example, when the conversion unit 10610 only performs an offset on the attribute information, it does not need to have the scale unit 10611.

[0239] The scale unit 10611 performs scaling (multiplication or division), which is an example of a conversion process, on the input attribute information and outputs a scale value (more specifically, scale information, which is information indicating the scale value, which is the value used for scaling).

[0240] The offset unit 10612 performs offset (addition or subtraction), which is another example of a conversion process, on the scaled attribute information and outputs an offset value (more specifically, offset information, which is information indicating the offset value, which is the value used for the offset).

[0241] The encoding unit 10620 encodes the attribute information converted by the conversion unit 10610 (post-conversion attribute information), and also encodes conversion information such as an offset value or a scale value as additional information (metadata).

[0242] The encoding unit 10620 includes an attribute information encoding unit 10621 and an additional information encoding unit 10622 .

[0243] The attribute information encoding unit 10621 encodes the converted attribute information, which is the attribute information converted by the conversion unit 10610.

[0244] The additional information encoding unit 10622 encodes the additional information including the conversion information such as the scale value and offset value output by the conversion unit 10610 .

[0245] For example, if the encoding unit 10620 does not support encoding of negative values, or if it is specified that the encoding unit 10620 does not support encoding of negative values, when the format of the input attribute information has a negative value, the conversion unit 10610 adds an offset value to the attribute information and converts the attribute information to a positive value.

[0246] For example, if the encoding unit 10620 does not support decimals or floating points but does support integers, or if the encoding unit 10620 is specified to not support decimals or floating points but does support integers, when the format of the input attribute information is not an integer, the scale unit 10611 multiplies the input attribute information (more specifically, the numerical value indicated by the input attribute information) by a scale value to convert the attribute information into a positive numerical value.

[0247] For example, if the encoding unit 10620 supports encoding of attribute information of 8-bit unsigned integer type (positive integer) and the input attribute information is a 32-bit signed floating point in the range of [-1, 1], the attribute information is first converted into a scaled_value, which is an 8-bit signed integer type value in the range of [-127, 128], by processing such as scaling, rounding, truncation, and rounding up.

[0248] Note that scaled_attribute=round(input_attribute×scale).

[0249] Here, scale is an example of a scale value, which is a value by which the value indicated by the attribute information is multiplied, for example, 2^(8bit-1), that is, 2 to the power of 7=128.

[0250] Next, the scaled attribute information is converted to an 8-bit unsigned integer type in the range of [0, 255] using an offset.

[0251] Note that offset_attribute=scaled_attribute+offset.

[0252] Here, offset is an example of an offset value, which is a value to be added to the value indicated by the attribute information, and is, for example, 2^(8bit-1)-1=127.

[0253] The offset value and / or scale value, which are the conversion information used in the conversion, are input to the encoding unit 10620 and encoded as side information.

[0254] The additional information encoding unit 10622 may encode the transformation information as it is as the additional information, or may encode information from which the transformation information can be derived as the additional information.

[0255] Furthermore, the information from which an offset value can be derived and the information from which a scale value can be derived may be shown independently, or may be shown as common information.

[0256] For example, in the above example, the offset value and scale value are determined in advance as offset=2^(N-1) and scale=2^(N-1)-1. In this case, for example, the encoding unit 10620 stores the value of N (an integer equal to or greater than 1) in the additional information and encodes it, that is, encodes the additional information indicating the value of N as conversion information.

[0257] Furthermore, if the encoding unit supports 8-bit unsigned integers, N may be defined as the number of bits of the unsigned integer, and N=8.

[0258] Alternatively, N may be predetermined to indicate the number of bits of an unsigned integer type. In such a case, when information indicating that N indicates the number of bits of an unsigned integer type is stored in the coded stream (bitstream), the information indicating that N indicates the number of bits of an unsigned integer type does not need to be included in the additional information.

[0259] Furthermore, the conversion unit 10610 may determine the offset and scale (offset value and scale value) based on the values ​​and features indicated by the attribute information constituting the three-dimensional point group.

[0260] After scaling the attribute information, the scale unit 10611 may round the value of the attribute information after scaling by rounding off, rounding down, rounding up, or the like.

[0261] Furthermore, if there is no need to convert the attribute information so that the value indicated by the attribute information is a positive integer, the conversion unit 10610 may not convert the attribute information. In this case, the conversion unit 10610 may not output the scale value and offset value, or may output information indicating that no conversion was performed as conversion information. In this case, for example, the encoding unit 10620 encodes the attribute information that has not been converted by the conversion unit 10610.

[0262] Fig. 34 is a block diagram showing the configuration of a three-dimensional data decoding device according to this embodiment. Note that Fig. 34 omits the illustration of a decoding unit that is provided in the three-dimensional data decoding device and that decodes encoded position information.

[0263] The three-dimensional data decoding device 10630 includes a decoding unit 10640 and an inverse transform unit 10650.

[0264] The decoding unit 10640 receives the coded attribute information (coded attribute information) and the coded additional information (coded additional information) and decodes the coded attribute information and the coded additional information. The decoding unit 10640 includes an attribute information decoding unit 10641 and an additional information decoding unit 10642.

[0265] The attribute information decoding unit 10641 generates decoded attribute information by decoding the encoded attribute information.

[0266] The additional information decoding unit 10642 extracts conversion information indicating an offset value, a scale value, and the like by decoding the coded additional information.

[0267] The inverse conversion unit 10650 performs inverse conversion processing on the decoded attribute information based on the conversion information. The inverse conversion processing is at least one of inverse offset (inverse offset processing) and inverse scaling (inverse scaling processing), which will be described later. The inverse conversion unit 10650 includes an inverse offset unit 10651 and an inverse scale unit 10652.

[0268] The reverse offset unit 10651 reverse-offsets the decoded attribute information, which is an example of reverse conversion processing, using an offset value extracted from the conversion information. That is, the reverse offset unit 10651 performs conversion on the decoded attribute information that is the reverse of the conversion that the conversion unit 10610 (more specifically, the offset unit 10612) performed on the attribute information. For example, when the conversion unit 10610 added an offset value to a value indicated by the attribute information, the reverse offset unit 10651 subtracts the offset value from the value indicated by the decoded attribute information.

[0269] The inverse scale unit 10652 inversely scales the inversely offset decoded attribute information, which is another example of inverse conversion processing, using the scale value extracted from the additional information. That is, the inverse scale unit 10652 performs a conversion on the decoded attribute information that is the inverse of the conversion that the conversion unit 10610 (more specifically, the scale unit 10611) performed on the attribute information. For example, if the conversion unit 10610 multiplied the value indicated by the attribute information by the scale value, the inverse scale unit 10652 divides the value indicated by the decoded attribute information by the scale value.

[0270] For example, if the offset value extracted from the additional information is offset and the scale value is scale, the inversely offset attribute information is derived as offset_attribute=decoded_value-offset, and the inversely scaled attribute information is derived as scaled_attribute=offset_attribute / scale.

[0271] In scaling and inverse scaling, the amount of processing may be reduced by using a shift operation (bit shift) instead of multiplication and / or division by expressing the scale value as a power of 2. In other words, scaling and inverse scaling are processes that perform at least one of multiplication, division, and shift operations on values ​​indicated by attribute information.

[0272] With the above configuration, the inverse conversion unit 10650 provided in the three-dimensional data decoding device 10630 performs an inverse conversion process based on the conversion information included in the encoded data, thereby making it possible to reproduce the attribute information before it was converted by the conversion unit 10610 provided in the three-dimensional data encoding device 10600.

[0273] The three-dimensional data decoding device 10630 does not necessarily have to perform the inverse transformation process, and may select whether or not to perform the inverse transformation process based on the application or use case.

[0274] In addition, although the conversion unit 10610 has been described as being configured such that the offset unit 10612 is located after (at a later stage of) the scale unit 10611, and the inverse conversion unit 10650 has been described as being configured such that the inverse offset unit 10651 is located after the inverse scale unit 10652, the configuration is not limited to this. For example, the conversion unit 10610 may be configured such that the scale unit 10611 is located after the offset unit 10612, and the inverse conversion unit 10650 may be configured such that the inverse offset unit 10651 is located after the inverse scale unit 10652.

[0275] Alternatively, the three-dimensional data encoding device 10600 may select which configuration to use, that is, the order in which to perform scaling and offsetting on the attribute information, based on the type of attribute information (attribute_type), and store information indicating which configuration was used, that is, information indicating the order in which scaling and offsetting were performed (order information), in the additional information, for example as a flag, and transmit this information to the three-dimensional data decoding device 10630. The three-dimensional data decoding device 10630 may select the order in which to perform inverse scaling and inverse offsetting based on the order information, and perform inverse transform processing on the decoded attribute information in the selected order.

[0276] FIG. 35 is a diagram showing a first example of syntax of an SPS (Sequence parameter set) according to the present embodiment.

[0277] In this embodiment, the SPS indicates an SPS identifier (sps_idx) and additional information (common_information()) related to the entire sequence. For example, for each attribute included in the bitstream, the SPS indicates additional information related to the attribute information, such as an attribute information identifier (attribute_type), the number of dimensions of the attribute information (num_dimension), an identifier for identifying instances of the same attribute type (instance_id), and other additional information related to the attribute information (attribute_info).

[0278] The three-dimensional point cloud data may not have attribute information corresponding to the position information, or may have one or more attribute information corresponding to the position information. For example, when the three-dimensional point cloud data has a plurality of attribute information corresponding to one piece of position information, the three-dimensional data encoding device 10600 generates conversion information corresponding to each of the attribute information and stores the generated conversion information in the additional information (in other words, generates additional information including the conversion information).

[0279] "transform_flag" is information (flag) indicating whether the additional information for each attribute information includes transformation information. For example, if the three-dimensional data encoding device 10600 includes transformation information in the additional information, it sets "transform_flag=1" and sets the transformation information (offset value and / or scale value). For example, if the three-dimensional data encoding device 10600 does not include transformation information, it sets "transform_flag=0."

[0280] The number of dimensions of the attribute information is 1 or more, and when the number of dimensions is 2 or more, common conversion information may be used for all dimensions. Of course, even when the number of dimensions of the attribute information is 2 or more, individual conversion information may be used for all dimensions.

[0281] For example, if the attribute information is composed of three-dimensional color information and one-dimensional reflectance, common conversion information may be applied to R (Red), G (Green), and B (Blue) in the color information, and conversion information different from that of the color information may be applied to the reflectance.

[0282] Furthermore, the conversion information may be used as common conversion information for a specific plurality of pieces of attribute information and stored in the additional information common to the specific plurality of pieces of attribute information. Alternatively, for example, the conversion information may be used as common conversion information for a plurality of instances of the same attribute type (attribute_type), and the conversion information may be shown for each type of attribute information.

[0283] Alternatively, it may be predetermined based on the type of attribute that conversion information is not used and is not indicated in additional information for a specific attribute type.

[0284] FIG. 36 is a diagram showing an example of the configuration of a bitstream according to this embodiment.

[0285] The conversion information is stored in, for example, the SPS.

[0286] The transformation information does not have to be stored in the SPS, but may be stored in a parameter set related to encoding of attribute information (APS / Attribute Parameter Set) or other additional information such as a slice header.

[0287] Alternatively, the conversion information may be stored in additional information such as SEI (Supplemental Enhancement Information).

[0288] Instance_id indicates an identifier of the same attribute type. For example, in the case of Color1, attribute_type=color is indicated and instance_id=0. Also, in the case of Color2, attribute_type=color is indicated and instance_id=1.

[0289] For example, if the attribute information is color information, no conversion information is generated.

[0290] Furthermore, for example, when the attribute information is information indicating a normal vector, conversion information is generated and included in the bitstream together with the SPS or APS.

[0291] For example, if Attribute0 included in the bitstream is color information (Color), which is an example of attribute information, the flag information (transform_flag) indicating whether transformation information is included in the bitstream is set to transform_flag=0, indicating that no transformation processing has been performed on Attribute0.

[0292] On the other hand, for example, if Attribute1 included in the bitstream is normal vector information (Normal_Vector), which is another example of attribute information, setting transform_flag=1 indicates that a transformation process has been performed on Attribute1. In this case, for example, the bitstream includes information indicating the value used in the offset (offset value) and information indicating the value used in the scaling (scale value), such as offset=A and scale=B.

[0293] Note that transform_flag may be replaced with transform_information_type, and the combination indicating the offset value and scale value may be switched based on transform_information_type, or the method and / or format for indicating the offset and / or scale value may be switched. As described above, the bitstream contains information (transform identification information) indicating whether or not a transform process has been performed, such as transform flag information like transform_flag and transform type information like transform_information_type.

[0294] Fig. 37 is a diagram showing a second example of the syntax of the SPS according to this embodiment, and Fig. 38 is a diagram showing a first example of the syntax of the conversion information according to this embodiment.

[0295] For example, as shown in Fig. 38, the method of indicating the scale value and offset value may be selectable from among showing the value as it is, showing it as a power of 2 (value calculated by y = log2(x)), showing it by a predetermined method, etc. For example, a flag indicating which syntax to use may be displayed, and the syntax may be switched by the flag.

[0296] For example, when transform_type==2, the three-dimensional data encoding device acquires offset_log2 and calculates offset=2^(offset_log2-X)-Y and scale=2^(scale_log2-Z) to calculate the offset value and scale value.

[0297] Note that X, Y, and Z are any integers. X, Y, and Z may be predetermined values, or information indicating these values ​​may be included in the bit stream and transmitted from the three-dimensional data encoding device 10600 to the three-dimensional data decoding device 10630.

[0298] In the above calculation, instead of offset_lo2 and scale_log2, values ​​obtained by subtracting 1 (for example, offset_log2_minus1 and scale_log2_minus_1) may be stored in the additional information.

[0299] This allows the number of bits to be reduced.

[0300] FIG. 39 is a diagram showing a second example of the syntax of the conversion information according to the present embodiment.

[0301] In the example shown in FIG. 39, the syntax is switched based on the type of attribute information (attribute_type), such as indicating either an offset value or a scale value, indicating both, or indicating neither.

[0302] When both offset and scale are performed, order information (transform_order) indicating the order of configuration of the offset unit 10612 and the scale unit 10611 of the transform unit 10610 included in the three-dimensional data encoding device 10600 may be included in the bitstream as additional information, for example. This enables the three-dimensional data decoding device 10630 to extract predetermined syntax based on the additional information such as transform_type, attribute_type, and transform_order, and apply it to the inverse transform unit 10650.

[0303] FIG. 40 is a flowchart showing the processing procedure of the three-dimensional data encoding device according to this embodiment.

[0304] First, the three-dimensional data encoding device 10600 determines whether or not to convert the input attribute information (S10601).

[0305] When it is determined that the input attribute information is to be converted (Yes in S10601), the three-dimensional data encoding device 10600 performs a conversion process on the input attribute information (S10602). For example, the three-dimensional data encoding device 10600 performs offset and scale on the input attribute information.

[0306] Next, the three-dimensional data encoding device 10600 stores the transformation information in the additional information and sets transform_flag=1 (S10603). For example, information indicating the offset value used for the offset and the scale value used for the scale is included in the additional information as the transformation information.

[0307] Next, the three-dimensional data encoding device 10600 encodes the additional information including the conversion information and the attribute information on which the conversion process has been performed (S10604). After step S10604, for example, the three-dimensional data encoding device 10600 generates a bit stream including the encoded information as encoded data and transmits it to the three-dimensional data decoding device 10630.

[0308] On the other hand, if the three-dimensional data encoding device 10600 determines that the input attribute information should not be converted (No in S10601), it does not perform conversion processing on the input attribute information, and therefore does not store the conversion information in the additional information and sets transform_flag=0 (S10605).

[0309] Next, the three-dimensional data encoding device 10600 encodes the additional information that does not include conversion information and the attribute information for which conversion processing has not been performed, that is, the input attribute information (S10606). After step S10606, for example, the three-dimensional data encoding device 10600 generates a bit stream that includes this encoded information as encoded data and transmits it to the three-dimensional data decoding device 10630.

[0310] FIG. 41 is a flowchart showing the processing procedure of the three-dimensional data decoding device according to this embodiment.

[0311] First, the three-dimensional data decoding device 10630 receives, for example, a bit stream transmitted by the three-dimensional data encoding device 10600, decodes the encoded data contained in the received bit stream, and analyzes the additional information contained in the decoded encoded data (S10611).

[0312] Next, the three-dimensional data decoding device 10630 decodes the attribute information of the coded data included in the bitstream (S10612).

[0313] Next, the three-dimensional data decoding device 10630 determines whether or not transform_flag included in the bitstream is set to 1 (S10613). In other words, the three-dimensional data decoding device 10630 determines whether or not transform processing has been performed on the attribute information included in the bitstream by determining whether or not transform_flag=1.

[0314] If the three-dimensional data decoding device 10630 determines that the transform_flag included in the bitstream is set to 1 (Yes in S10613), it extracts transformation information from the additional information and performs inverse transformation processing on the attribute information based on the extracted transformation information (S10614).

[0315] On the other hand, if the three-dimensional data decoding device 10630 determines that the transform_flag included in the bitstream is not set to 1 (No in S10613), that is, if transform_flag = 0, it terminates the processing without performing the inverse transform process.

[0316] In the above example, the data format of the attribute information input to the encoding unit 10620 is converted and the conversion information is included in the bitstream, but the present invention is not limited to this.

[0317] Fig. 42 is a block diagram for explaining another example of the processing of the three-dimensional data encoding device according to this embodiment. Fig. 43 is a block diagram for explaining another example of the processing of the three-dimensional data decoding device according to this embodiment.

[0318] For example, the three-dimensional data encoding device may store format information indicating the data format of the attribute information input to the conversion unit 10660 and the format of the attribute information to be encoded after conversion in the additional information, and encode the additional information including the format information in the encoding unit 10670.

[0319] The three-dimensional data decoding device 10630 can reproduce the attribute information before the conversion process in the conversion unit 10660 provided in the three-dimensional data encoding device by performing an inverse conversion on the decoded attribute information in the inverse conversion unit 10690 based on the format information extracted by the decoding unit 10680.

[0320] Format information (data format information) is information indicating, for example, the data type, the number of bits, signed or unsigned, etc. For example, format information is information such as int8, uint16, float16, etc. The number such as 8 in int8 indicates the number of bits.

[0321] Alternatively, for example, the format information may indicate the file format of the point cloud data before the conversion process is performed (e.g., a ply file, a pcd file, a Las file, a txt file, a csv file, etc.), or may indicate the file format of the point cloud data after the conversion process is performed.

[0322] Alternatively, for example, the format information may indicate at least one of the file format of the point cloud data before the conversion process is performed and the file format of the point cloud data after the conversion process is performed as extended information SEI. Furthermore, the SEI may include header information included in each file format.

[0323] Here, file formats of point cloud data include, for example, the ply format, which is a polygon file format, the las format, which is a format for LiDAR data obtained by laser surveying, and the pcd format, which is a point cloud file format.

[0324] Furthermore, at least one of the file format and header information may be included in user_data.

[0325] This allows the three-dimensional data encoding device to encode the attribute information using the format information extracted by the conversion unit 10660. Alternatively, the three-dimensional data decoding device can inversely convert the decoded attribute information using the format information, and can also reconstruct header information and the like that is not to be encoded.

[0326] Furthermore, although the offset and scale of the attribute information have been described above, the method described in this embodiment can also be applied to position information.

[0327] Before encoding the position information, conversion processing such as offset and scale may be performed, and after decoding the position information, inverse conversion processing may be performed. In this case, the conversion information or format information may be stored in additional information such as SPS.

[0328] Furthermore, the three-dimensional data encoding device may have a conversion unit that performs conversion on at least one or both of the position information and the attribute information. Similarly, the three-dimensional data decoding device may have an inverse conversion unit that performs conversion on either one or both (i.e., at least one) of the position information and the attribute information. In such cases, either one or both (i.e., at least one) of the conversion information for the position information and the conversion information for the attribute information may be included in the additional information.

[0329] Although offsetting, scaling, and quantization have been described above as methods for converting input point cloud data, other conversion methods may be used instead. For example, the conversion may use a predetermined linear or nonlinear conversion means, such as a conversion or approximation using a predetermined function.

[0330] The additional information may include not only information indicating the format of the attribute information, but also information indicating the order of the point cloud data, information indicating the sort order, time stamp information, and the like.

[0331] (Fourth embodiment) A specific example of SEI (Supplemental Enhancement Information) which is extension information will be described below. Fig. 44 is a diagram showing an example of the syntax of SEI.

[0332] The SEI includes format_id, which indicates the format type. For example, format_id=0 indicates the ply format, format_id=1 indicates the las format, and format_id=2 indicates the pcd format. Note that the correspondence between the value set in format_id and the format is not limited to the above example, and can be set arbitrarily. For example, other commonly known point cloud formats may be defined as format_id.

[0333] Furthermore, the SEI may include at least a portion of the header information of the format depending on the format_id. For example, when format_id=0, the SEI includes ply_format_info(), which includes the header information of the ply format. When format_id=1, the SEI includes las_format_info(), which includes the header information of the las format. The SEI may also include the header information of other commonly known point cloud formats.

[0334] Figure 45 is a diagram showing an example of the syntax of ply_format_info(). ply_format_info() includes information included in the header of the ply format or part of that information. For example, as shown in Figure 45, ply_format_info() includes format_info, element_info, property_num, property_info, property_type, scale, offset, is_property, fill_property, sps_id, and component_id.

[0335] format_info indicates whether the data is in binary format or text format. If the data is in binary format, ply_format_info() may include information indicating whether the data is in big endian or little endian format.

[0336] The element_info indicates the type of data (for example, a vertex indicating a point that constitutes a three-dimensional point or three-dimensional polygon data, etc.) The ply_format_info() may include information indicating the number of pieces of data.

[0337] The property_num field indicates the number of properties (also called components) included in the ply format. The ply_format_info() field contains information for each property, such as property_info, property_type, scale, offset, is_property, fill_property, sps_id, and component_id.

[0338] The property_info indicates the type of property and includes a property name or identifier, etc. Specifically, the type of property is each element of position information (x coordinate, y coordinate, z coordinate in a Cartesian coordinate system, or distance, horizontal angle, elevation angle in a polar coordinate system), each element of color information (R, G, B), reflectance, or other information element.

[0339] The property_type indicates the data type such as float, int, unsigned int, etc. The scale indicates the scale value used in scaling processing. The offset indicates the offset value used in offset processing.

[0340] "is_property" indicates whether the property is to be compressed. For example, if the property is to be compressed, is_property=1 is set, and if the property is not to be compressed, is_property=0 is set.

[0341] If the property is to be compressed, ply_format_info() includes information for identifying the compressed data of the encoded data, such as sps_id, which is the identifier of the SPS, and component_id, which is the identifier of the property (component).

[0342] For example, the property is associated with the attribute information described in the component_id-th position among a plurality of pieces of attribute information described in an SPS (Sequence Parameter Set) having the specified sps_id.

[0343] If the property is not subject to compression, ply_format_info() includes fill_property, which indicates the value to be restored as the data value of that property during decoding. If different values ​​are required for each point, ply_format_info() may include as many fill_property elements as there are pieces of data.

[0344] That is, if a property (component) is to be decoded, the property data is composed of the decoded data specified by sps_id and component_id. If the property is not to be decoded, the property data is composed using the value indicated by fill_property. In addition, in the syntax, fill_property indicates an example of a value that is commonly used for all points included in the point cloud, but it may also indicate a different value for each point.

[0345] Additionally, ply_format_info() may include information about other properties as needed.

[0346] 46 is a diagram showing an example of the syntax of las_format_info(). las_format_info() includes information contained in the header of the las format. For example, las_format_info includes public_header_block(), variable_length_records(), point_data_records(), and extended_variable_length_records().

[0347] The public_header_block() contains header information about the data stored in Point Data Records in las format.

[0348] variable_length_records() contains any variable length header information stored in Variable Length Records in las format.

[0349] point_data_records() contains information about the location or attribute information of points stored in Point Data Records in las format.

[0350] extended_variable_length_records() contains extended information at the end of the las format, which is stored in Extended Variable Length Records in the las format.

[0351] 47 is a diagram showing an example of the syntax of public_header_block(). public_header_block() may include the information included in the Public Header Block in the las format as is, or may include only a part of it.

[0352] 48 is a diagram showing an example of the syntax of variable_length_records(). variable_length_records() may include the information included in the Variable Length Record Header in the las format as is, or may include a part of it.

[0353] 49 is a diagram showing an example of the syntax of point_data_records(). point_data_records may include information about the position information or attribute information of the point cloud included in Point Data Records in the las format. For example, point_data_records() includes component_num, component_info, scale, offset, is_component, and fill_component.

[0354] component_num indicates the number of properties (components) for one point included in the las format. Also, point_data_records() includes information for each property that conforms to the Point Data Record Format defined in the las format (component_info, scale, offset, is_component, fill_component). Note that point_data_records() may include information about all properties, or may include information about some of the properties.

[0355] The component_info is information indicating the correspondence between the property and the actual data, and specifically includes the property name or the property ID (identifier).

[0356] Additionally, point_data_records() may include information for each property, the number of which is indicated by property_num. For example, this information includes scale, offset, is_component, and fill_component.

[0357] scale indicates the scale value used in scaling. offset indicates the offset value used in offset processing. is_component indicates whether the target property is to be compressed. If the target property is not to be compressed, point_data_records() includes fill_component. fill_component indicates the value to be restored as the data value of that property when decoding. Also, if restoration with a different value for each point is required, point_data_records() may include as many fill_components as there are pieces of data.

[0358] Additionally, point_data_records() may include information about other properties as needed.

[0359] 50 is a diagram showing an example of the syntax of extended_variable_length_records(). extended_variable_length_records() may include the information included in Extended Variable Length Records in the las format as is, or may include a part of it.

[0360] A first example of the transform unit and inverse transform unit will be described below. Fig. 51 is a block diagram showing the configuration of a three-dimensional data encoding device 13900 according to the first example. Note that this figure mainly shows processing units related to encoding attribute information, and omits processing units related to encoding position information. The three-dimensional data encoding device 13900 includes a transform unit 13901 and an encoding unit 13902.

[0361] The conversion unit 13901 generates converted attribute information by converting attribute information included in the point cloud data. The conversion unit 13901 includes a scale unit 13903 and an attribute information precision determination unit 13904.

[0362] The attribute information precision determination unit 13904 determines the precision of the attribute information to be coded. Specifically, the attribute information precision determination unit 13904 determines the valid bits from the multiple bits included in the attribute information, and determines a scale value (scale) to be used for division, multiplication, or bit shift to extract the valid bits.

[0363] The scaling unit 13903 generates converted attribute information by scaling (multiplying or dividing) the attribute information using the scale value determined by the attribute information precision determination unit 13904. In addition, the conversion unit 13901 outputs conversion information including the scale value to the encoding unit 13902.

[0364] The encoding unit 13902 includes an attribute information encoding unit 13905 and an additional information encoding unit 13906. The attribute information encoding unit 13905 generates encoded attribute information by encoding converted attribute information. The additional information encoding unit 13906 generates encoded additional information by encoding conversion information including a scale value as additional information (also referred to as metadata or control information). This encoded attribute information and encoded additional information are included in a bit stream (also referred to as an encoded bit stream or encoded data) output by the three-dimensional data encoding device 13900.

[0365] 52 is a block diagram showing the configuration of a three-dimensional data decoding device 13910 according to the first example. Note that the diagram mainly shows processing units related to decoding of attribute information, and omits processing units related to decoding of position information.

[0366] This three-dimensional data decoding device 13910 decodes a bit stream including coded attribute information and coded additional information that is generated by, for example, the three-dimensional data coding device 13900 shown in Fig. 51. The three-dimensional data decoding device 13910 includes a decoding unit 13911 and an inverse conversion unit 13912.

[0367] The decoding unit 13911 includes an attribute information decoding unit 13913 and an additional information decoding unit 13914. The attribute information decoding unit 13913 generates decoded attribute information by decoding the coded attribute information. The additional information decoding unit 13914 generates conversion information including a scale value by decoding the coded additional information.

[0368] The inverse conversion unit 13912 includes an inverse scale unit 13915. The inverse scale unit 13915 generates attribute information by inversely scaling the decoded attribute information using the scale value.

[0369] A specific example of processing by the conversion unit 13901 in the first example will be described below. The conversion unit 13901 checks the positions of the valid bits of all attribute information to be coded (attribute information at points 0 to n). FIGS. 53 and 54 are diagrams showing an example of processing by the conversion unit 13901. In the example shown in FIG. 53, the bit depth of the attribute information is 16 bits. Here, the attribute information is expressed in 16 bits, with the upper 8 bits being valid bits and the lower 8 bits being invalid bits. Note that bit depth is sometimes expressed as bit width or bit precision. In this case, the conversion unit 13901 performs scaling processing on the attribute information by dividing by scale=256, multiplying by scale=1 / 256, or performing a right shift operation with scale=8 bits. As a result, the lower 8 bits are validated, and converted attribute information with a bit depth of 8 bits is generated.

[0370] In the example shown in FIG. 54, the bit depth of the attribute information is 16 bits, with the lower 8 bits being valid bits and the upper 8 bits being invalid bits. In this case, the conversion unit 13901 performs a scaling process on the attribute information by dividing by scale=1, multiplying by scale=1, or performing a right shift operation with scale=0 bits. This validates the lower 8 bits, and generates converted attribute information with a bit depth of 8 bits. Note that in this case, the conversion unit 13901 may not perform scaling, but may instead clip the upper 8 bits and validate the lower 8 bits to generate attribute information with a bit depth of 8 bits.

[0371] In this way, the conversion unit 13901 converts attribute information with a bit depth of 16 bits into converted attribute information of 8 bits. Therefore, the encoding unit 13902 can handle values ​​smaller than 16-bit attribute information, thereby improving compression efficiency.

[0372] It is not necessary for the conversion unit 13901 to check the precision of the attribute information. For example, if the position of the significant bits of the attribute information is determined by the format standard of the attribute information to be encoded, the conversion unit 13901 may determine the scale value (scale) according to that rule.

[0373] Also, for example, if the lower 8 bits are valid bits, the conversion unit 13901 may output 16-bit attribute information in which the upper 8 bits remain invalid (filled with 0) as converted attribute information without changing the bit depth to 8 bits.

[0374] Furthermore, although the above specific example has been described with respect to 16-bit attribute information, the same technique can be applied to attribute information of any bit depth.

[0375] The conversion unit 13901 may further include an offset unit that performs offset processing. In this case, the offset processing may be performed after or before the scaling processing.

[0376] Furthermore, the conversion unit 13901 may change the scale value for each coding group unit such as a slice, a tile, or a frame. Furthermore, the conversion unit 13901 may change the scale value for each type of attribute information. This allows the conversion unit 13901 to use different scale values ​​between units or types.

[0377] A specific example of processing by the inverse conversion unit 13912 in the first example will be described below. Figures 55 and 56 are diagrams showing examples of processing by the inverse conversion unit 13912. In the example shown in Figure 55, the bit depth of the decoded attribute information is 8 bits. The inverse conversion unit 13912 performs multiplication by scale=256 or a left shift operation with scale=8 bits on the decoded attribute information to generate attribute information with a bit depth of 16 bits in which the most significant 8 bits are valid.

[0378] 56, the bit depth of the decoded attribute information is 8 bits. The inverse conversion unit 13912 performs multiplication by scale=1 or a left-shift operation with scale=0 on the decoded attribute information to generate attribute information with a bit depth of 16 bits in which the lower 8 bits are valid. In this case, the inverse conversion unit 13912 may not perform scaling processing, but may instead validate the lower 8 bits to generate attribute information with a bit depth of 16 bits.

[0379] In this way, the three-dimensional data decoding device 13910 can restore the original attribute information by decoding the bit stream generated by the above-mentioned three-dimensional data encoding device 13900. In other words, the three-dimensional data decoding device 13910 can decode a bit stream with improved compression efficiency.

[0380] As described above, the inverse conversion unit 13912 included in the three-dimensional data decoding device 13910 performs inverse conversion processing based on the conversion information included in the bitstream, thereby restoring the attribute information before it was converted by the conversion unit 13901 included in the three-dimensional data encoding device 13900. Note that the three-dimensional data decoding device 13910 does not necessarily have to perform inverse conversion processing, and may select whether or not to perform inverse conversion processing based on the application or use case.

[0381] Although the above specific example describes the case of 16-bit attribute information, the same technique can be applied to attribute information of any bit depth.

[0382] The inverse conversion unit 13912 may further include an inverse offset unit that performs inverse offset processing. In this case, the inverse scaling processing may be performed after or before the inverse offset processing.

[0383] Furthermore, the inverse transform unit 13912 may perform inverse transform by changing the scale value for each coding group unit such as a slice, a tile, or a frame. Furthermore, the inverse transform unit 13912 may change the scale value for each type of attribute information. This allows the inverse transform unit 13912 to use different scale values ​​between units or types.

[0384] Modified examples of the syntax will be described below. Fig. 57 is a diagram showing an example of the syntax of an SPS (Sequence Parameter Set). The SPS includes sps_idx, common_information(), attribute_type, instance_id, num_dimension, num_attribute_parameter, attribute_parameter(i), and attribute_info().

[0385] sps_idx is the identifier of the SPS. common_information() is additional information related to the entire sequence. attribute_type, instance_id, num_dimension, num_attribute_parameter, and attribute_parameter(i) are set for each attribute information.

[0386] attribute_type is the identifier of the attribute information. instance_id is an identifier for identifying instances of the same attribute type. num_dimension indicates the number of dimensions of the attribute information. attribute_info() is other additional information related to the attribute information.

[0387] The attribute_parameter() includes various types of additional information (metadata) for each attribute information and has a general-purpose format. The num_attribute_parameter indicates the number of attribute_parameter().

[0388] 58 is a diagram illustrating an example of the syntax of attribute_parameter(i). The attribute_parameter(i) includes attr_param_type, attribute_source_offset_num_bits, attribute_source_offset, attribute_source_scale_num_bits, attribute_source_scale, and attribute_scale_frac_bits.

[0389] attr_param_type indicates the type of attribute_parameter. For example, when attr_param_type=3, attribute_parameter(i) includes attribute_source_offset_num_bits, attribute_source_offset, attribute_source_scale_num_bits, attribute_source_scale, and attribute_scale_frac_bits. This is the same as indicating transform_info when transform_flag=1.

[0390] The attribute_source_offset indicates an offset value for restoring the decoded attribute information to the original (source) attribute information.

[0391] The attribute_source_scale indicates a scale value for restoring the decoded attribute information to the original attribute information.

[0392] attribute_source_offset_num_bits indicates the number of bits of the offset value (attribute_source_offset). If the number of bits indicated by attribute_source_offset_num_bits is 0, it means that no offset processing is performed (i.e., offset = 0). In other words, attribute_source_offset_num_bits has the function of a flag that indicates whether or not to indicate an offset value (whether or not to perform offset processing).

[0393] attribute_source_scale_num_bits indicates the number of bits of the scale value (attribute_source_scale). If the number of bits indicated by attribute_source_scale_num_bits is 0, it means that no scaling processing is performed (i.e., scale = 1). In other words, attribute_source_scale_num_bits has the function of a flag that indicates whether or not to indicate a scale value (whether or not to perform scaling processing).

[0394] The attribute_scale_frac_bits indicates the number of bits required to represent the decimal point of the scale value. The scale value is calculated using this attribute_scale_frac_bits and the following (Equation a1).

[0395]

number

[0396] Next, we will explain the flow of the three-dimensional data encoding device 13900 and the three-dimensional data decoding device 13910. Figure 59 is a flowchart of the three-dimensional data encoding process by the three-dimensional data encoding device 13900.

[0397] First, the three-dimensional data encoding device 13900 determines whether or not to convert attribute information (S13901). For example, the three-dimensional data encoding device 13900 determines whether or not to convert attribute information based on an external instruction or setting. Note that the three-dimensional data encoding device 13900 may determine whether or not to convert attribute information based on the attribute information to be encoded, point cloud data to be encoded, etc.

[0398] When converting attribute information (Yes in S13901), the three-dimensional data encoding device 13900 checks the accuracy and valid bit positions of the attribute information of all three-dimensional points included in the point cloud data to be encoded (S13902). Next, the three-dimensional data encoding device 13900 determines a scale value based on the valid bit positions (S13903). Next, the three-dimensional data encoding device 13900 generates converted attribute information by converting the attribute information using the determined scale value (S13904). Next, the three-dimensional data encoding device 13900 stores conversion information including the scale value in the additional information (S13905). For example, the three-dimensional data encoding device 13900 stores num_attribute_parameter=1 and attr_param_type=3 in the additional information. In addition, the scale value is stored in attribute_source_scale. Specifically, the scale value is indicated using attribute_source_scale_num_bits, attribute_source_scale, and attribute_scale_frac_bits. Next, the three-dimensional data encoding device 13900 generates a bit stream (encoded data) by encoding the additional information including the scale value and the converted attribute information (S13906).

[0399] On the other hand, if the attribute information is not to be converted (No in S13901), the three-dimensional data encoding device 13900 does not store the conversion information in the additional information (S13907). For example, the three-dimensional data encoding device 13900 stores num_attribute_parameter=0 in the additional information. Next, the three-dimensional data encoding device 13900 generates a bit stream (encoded data) by encoding the additional information that does not include the conversion information and the attribute information (S13908).

[0400] 60 is a flowchart of three-dimensional data decoding processing by the three-dimensional data decoding device 13910. First, the three-dimensional data decoding device 13910 decodes the coded additional information included in the bitstream and analyzes the obtained additional information (S13911). Next, the three-dimensional data decoding device 13910 generates decoded attribute information by decoding the coded attribute information included in the bitstream (S13912).

[0401] Next, the three-dimensional data decoding device 13910 determines whether or not to inversely convert the attribute information (whether or not the attribute information has been converted) based on the information included in the additional information (S13913). For example, if num_attribute_parameter==1 and attr_param_type==3, the three-dimensional data decoding device 13910 determines to inversely convert the attribute information, and otherwise determines not to inversely convert the attribute information.

[0402] If it is determined that the attribute information is to be inversely transformed (Yes in S13913), the three-dimensional data decoding device 13910 generates attribute information by inversely transforming the decoded attribute information based on the scale value included in the additional information (S13914). On the other hand, if it is determined that the attribute information is not to be inversely transformed (No in S13913), the three-dimensional data decoding device 13910 does not perform inverse transformation, and outputs the decoded attribute information as attribute information.

[0403] Next, a second example of the transform unit and inverse transform unit will be described. Fig. 61 is a block diagram showing the configuration of a three-dimensional data encoding device 13900A according to the second example. Note that the figure mainly shows processing units related to encoding attribute information, and omits processing units related to encoding position information.

[0404] 61 differs from the three-dimensional data encoding device 13900 shown in Fig. 51 in that the configuration of a conversion unit 13901A is different from that of the conversion unit 13901. Specifically, the conversion unit 13901A includes an attribute information resolution determination unit 13904A instead of the attribute information accuracy determination unit 13904.

[0405] The attribute information resolution determination unit 13904A determines the resolution of the attribute information to be coded. The attribute information resolution determination unit 13904A determines how many times the value of the attribute information is greater than that of a reference signal, and determines a scale value (scale) to be used for division, multiplication, or bit shifting to extract the reference signal.

[0406] The scaling unit 13903 generates converted attribute information by scaling (multiplying or dividing) the attribute information using the scale value determined by the attribute information resolution determining unit 13904 A. The converting unit 13901 A outputs conversion information including the scale value to the encoding unit 13902.

[0407] The encoding unit 13902 includes an attribute information encoding unit 13905 and an additional information encoding unit 13906. The attribute information encoding unit 13905 generates encoded attribute information by encoding transformed attribute information. The additional information encoding unit 13906 generates encoded additional information by encoding transformation information including a scale value as additional information. This encoded attribute information and encoded additional information are included in the bitstream output by the three-dimensional data encoding device 13900A.

[0408] The configuration of three-dimensional data decoding device 13910 that decodes the bit stream generated by three-dimensional data encoding device 13900A is the same as the configuration shown in FIG.

[0409] A specific example of processing by the conversion unit 13901A in the second example will be described below. The conversion unit 13901A checks the resolution of all attribute information to be coded (attribute information at points 0 to n). Fig. 62 is a diagram showing an example of processing by the conversion unit 13901A. In the example shown in Fig. 62, the bit depth of the attribute information is 16 bits, and the attribute information is in increments of 255, that is, a multiple of 255. In this case, the conversion unit 13901A performs scaling processing on the attribute information by dividing by scale=255 or multiplying by scale=1 / 255, thereby generating converted attribute information with a bit depth of 8 bits.

[0410] In this way, the conversion unit 13901A can convert attribute information with a bit depth of 16 bits into converted attribute information of 8 bits. Therefore, the encoding unit 13902 can handle values ​​smaller than 16-bit attribute information, thereby improving compression efficiency.

[0411] Note that the conversion unit 13901A does not need to check the accuracy of the attribute information. For example, if the resolution of the attribute information is known based on the format standard of the attribute information to be encoded, the conversion unit 13901A may determine the scale value (scale) according to that rule.

[0412] Furthermore, even if the attribute information is in the 8-bit range after scaling, the conversion unit 13901A may output 16-bit attribute information with the upper 8 bits remaining invalid (filled with 0) as the converted attribute information without changing the bit depth to 8 bits.

[0413] Furthermore, although the above specific example describes the case of 16-bit attribute information, the same argument can be applied to attribute information of any bit depth.

[0414] Furthermore, the conversion unit 13901A may further include an offset unit that performs offset processing. In this case, the offset processing may be performed after or before the scaling processing.

[0415] Furthermore, the conversion unit 13901A may change the scale value for each coding group unit, such as a slice, a tile, or a frame. Furthermore, the conversion unit 13901A may change the scale value for each type of attribute information. This allows the conversion unit 13901A to use different scale values ​​between units or types.

[0416] A specific example of processing by the inverse conversion unit 13912 in the second example will be described below. Fig. 63 is a diagram showing an example of processing by the inverse conversion unit 13912. In the example shown in Fig. 63, the bit depth of the decoded attribute information is 8 bits. The inverse conversion unit 13912 multiplies the decoded attribute information by scale=255 to generate attribute information with a bit depth of 16 bits in which the most significant 8 bits are valid.

[0417] In this way, the three-dimensional data decoding device 13910 can restore the original attribute information by decoding the bit stream generated by the above-mentioned three-dimensional data encoding device 13900A. In other words, the three-dimensional data decoding device 13910 can decode a bit stream with improved compression efficiency.

[0418] As described above, the inverse conversion unit 13912 included in the three-dimensional data decoding device 13910 performs inverse conversion processing based on the conversion information included in the bitstream, thereby restoring the attribute information to its state before conversion by the conversion unit 13901A included in the three-dimensional data encoding device 13900A. The three-dimensional data decoding device 13910 does not necessarily have to perform inverse conversion processing, and may select whether or not to perform inverse conversion processing based on the application or use case.

[0419] Although the above specific example describes the case of 16-bit attribute information, the same technique can be applied to attribute information of any bit depth.

[0420] The inverse conversion unit 13912 may further include an inverse offset unit that performs inverse offset processing. In this case, the inverse scaling processing may be performed after or before the inverse offset processing.

[0421] Furthermore, the inverse transform unit 13912 may perform inverse transform by changing the scale value for each coding group unit such as a slice, a tile, or a frame. Furthermore, the inverse transform unit 13912 may change the scale value for each type of attribute information. This allows the inverse transform unit 13912 to use different scale values ​​between units or types.

[0422] In addition, in the above explanation, an example was shown in which the conversion unit 13901A divides the attribute information by the value 255, scale=255 is stored in the additional information, and the inverse conversion unit 13912 multiplies the attribute information by the value 255 analyzed from the additional information, but the three-dimensional data encoding device 13900A may also store the value by which the inverse conversion unit 13912 divides the attribute information in the additional information.

[0423] Furthermore, the processing procedure by the three-dimensional data encoding device 13900A of the second example is generally similar to the flowchart of the first example shown in Fig. 59. In the second example, in step S13902, the three-dimensional data encoding device 13900A investigates the resolution of the attribute information of all three-dimensional points included in the point cloud data to be encoded. Furthermore, the processing procedure by the three-dimensional data decoding device 13910 of the second example is the same as the flowchart of the first example shown in Fig. 60.

[0424] A third example of the transformation unit and inverse transformation unit will be described below. Fig. 64 is a block diagram showing the configuration of a three-dimensional data encoding device 13900B according to the third example. Note that this figure mainly shows processing units related to encoding attribute information, and omits processing units related to encoding position information.

[0425] 64 differs from the three-dimensional data encoding device 13900 shown in FIG. 51 in that the configuration of a conversion unit 13901B is different from that of the conversion unit 13901. Specifically, the conversion unit 13901B includes an attribute information range determination unit 13904B instead of the attribute information accuracy determination unit 13904. The conversion unit 13901B also includes an offset unit 13907.

[0426] The attribute information range determination unit 13904B determines the range of values ​​of attribute information to be coded. If the range of attribute information values ​​exceeds the system's allowable range or if it is determined that efficiency is poor, the attribute information range determination unit 13904B determines to perform at least one of scaling processing and offset processing. Furthermore, the attribute information range determination unit 13904B determines at least one of a scale value (scale) used for division, multiplication, or bit shifting, and an offset value (offset) used for addition or subtraction, so that the range of converted attribute information values ​​falls within the system's allowable range or so that efficiency is improved. Here, a case where efficiency is poor is a case where the amount of data can be reduced by scaling or offset processing, for example, a case where the amount of data can be reduced by offset processing as shown in the example shown in FIG. 67, which will be described later.

[0427] The scale unit 13903 generates scaled attribute information by scaling (multiplying or dividing) the attribute information using the scale value determined by the attribute information range determination unit 13904 B. The offset unit 13907 generates converted attribute information by offsetting (adding or subtracting) the scaled attribute information using the offset value determined by the attribute information range determination unit 13904 B.

[0428] The encoding unit 13902 includes an attribute information encoding unit 13905 and an additional information encoding unit 13906. The attribute information encoding unit 13905 generates encoded attribute information by encoding transformed attribute information. The additional information encoding unit 13906 generates encoded additional information by encoding transformation information including a scale value and an offset value as additional information. This encoded attribute information and encoded additional information are included in the bitstream output by the three-dimensional data encoding device 13900B.

[0429] The configuration of the three-dimensional data decoding device 13910 that decodes the bitstream generated by the three-dimensional data encoding device 13900B is generally the same as the configuration shown in Fig. 52. However, the inverse conversion unit 13912 generates attribute information by performing inverse offset processing in addition to inverse scaling processing on the decoded attribute information.

[0430] A specific example of processing by the conversion unit 13901B in the third example will be described below. The conversion unit 13901B checks the range of values ​​of all attribute information to be coded (attribute information at points 0 to n). FIGS. 65, 66, and 67 are diagrams showing an example of processing by the conversion unit 13901B. The example shown in FIG. 65 is a case where the bit depth that the system can support is 16 bits, and the maximum bit depth of the attribute information is 17 bits. In this case, the bit depth of the attribute information overflows by 1 bit with respect to the range supported by the system. Therefore, the conversion unit 13901B generates converted attribute information with a bit depth of 16 bits by dividing the attribute information by scale=2, multiplying it by scale=1 / 2, or right-shifting it by scale=1 bit so that the value of the converted attribute information falls within the range.

[0431] 66 shows an example in which the bit depth that the system can support is 16 unsigned bits, and the minimum value of the attribute information is −5027. In this case, the value of the attribute information exceeds the range supported by the system by 5027 in the negative direction. Therefore, the conversion unit 13901B adds an offset of 5027 to the attribute information so that the value of the converted attribute information falls within the range, thereby generating converted attribute information with a bit depth of 16 unsigned bits.

[0432] 67 shows an example in which the values ​​that the system can support are unsigned integer values ​​and the attribute information is a 64-bit floating point. In this case, the conversion unit 13901B multiplies the attribute information by scale=100000 to convert the attribute information into an integer value. Furthermore, to reduce the amount of data, the conversion unit 13901B adds offset=-2312000 to the attribute information after multiplication. As a result, attribute information with a bit depth of 8 unsigned bits is generated.

[0433] In this way, the three-dimensional data encoding device 13900B can convert attribute information having values ​​that exceed the range that the system can handle into converted attribute information having values ​​within the range by using scaling or offset processing.

[0434] The three-dimensional data encoding device 13900B may perform scaling processing for the purpose of quantization to reduce the amount of data. In this case, the larger the scale value, the larger the quantization error in the inverse transform, but the amount of data can be reduced accordingly.

[0435] In addition, the conversion unit 13901B may not investigate the range of the attribute information, but may determine the scale value and offset value according to the rules if the range of the attribute information is determined, for example, by the format standard of the attribute information to be encoded.

[0436] Furthermore, although the above specific examples have been described in connection with 16-bit or 64-bit floating-point attribute information, the same techniques can be applied to attribute information of any bit depth.

[0437] Furthermore, when performing a process equivalent to division in scaling, the conversion unit 13901B may perform rounding and adding. Here, rounding and adding is a fractional process in which the number of bits at the end is rounded down or rounded up.

[0438] Furthermore, although the offset process is performed after the scaling process in the above example, the scaling process may be performed after the offset process.

[0439] Furthermore, the conversion unit 13901B may change the scale value and offset value for each coding group unit such as a slice, a tile, or a frame. Furthermore, the conversion unit 13901B may change the scale value and offset value for each type of attribute information. This allows the conversion unit 13901B to use different scale values ​​between units or types.

[0440] A specific example of processing by the inverse conversion unit 13912 in the third example will be described below. Fig. 68, Fig. 69, and Fig. 70 are diagrams showing examples of processing by the inverse conversion unit 13912. In the example shown in Fig. 68, the inverse conversion unit 13912 inversely converts decoded attribute information having a bit depth of 16 bits using a scale value included in the additional information. Specifically, the inverse conversion unit 13912 generates attribute information having a bit depth of 17 bits by multiplying the decoded attribute information by scale=2 or performing a left shift operation by scale=1 bit.

[0441] 69, the bit depth of the decoded attribute information is 16 unsigned bits. The inverse conversion unit 13912 inversely converts the decoded attribute information using an offset value included in the additional information. Specifically, the inverse conversion unit 13912 adds offset=−5027 to the decoded attribute information to generate attribute information with a bit depth of 16 signed bits.

[0442] 70, the bit depth of the decoded attribute information is unsigned 8 bits. The inverse conversion unit 13912 inversely converts the decoded attribute information using the offset value and scale value included in the additional information. Specifically, the inverse conversion unit 13912 adds offset=2312000 to the decoded attribute information, and multiplies the added value by scale=1 / 100000 to generate 64-bit floating-point attribute information.

[0443] In this way, the three-dimensional data encoding device 13900B can convert attribute information having values ​​outside the range that the system can handle into converted attribute information having values ​​within the range by scaling or offsetting. Furthermore, the three-dimensional data decoding device 13910 can decode the compressed data generated in this way to restore the original attribute information.

[0444] Furthermore, scaling processing may be performed for the purpose of quantization to reduce the amount of data. In this case, the larger the scale value, the larger the quantization error in the attribute information restored by inverse transformation, but the amount of data can be reduced accordingly.

[0445] As described above, the inverse conversion unit 13912 included in the three-dimensional data decoding device 13910 performs inverse conversion processing based on the conversion information included in the bitstream, thereby restoring the attribute information before conversion by the conversion unit 13901B included in the three-dimensional data encoding device 13900B. The three-dimensional data decoding device 13910 does not necessarily have to perform inverse conversion processing, and may select whether or not to perform inverse conversion processing based on the application or use case.

[0446] Although the above specific example describes the case of 16-bit attribute information, a similar technique can be applied to attribute information of any bit depth. Furthermore, when performing a process equivalent to division in the inverse scaling process, the three-dimensional data decoding device 13910 may perform rounding and addition.

[0447] Additionally, the inverse scaling process may be performed after or before the inverse offset process.

[0448] Furthermore, the inverse transform unit 13912 may perform inverse transform by changing the scale value and offset value for each coding group unit such as a slice, a tile, or a frame. Furthermore, the inverse transform unit 13912 may change the scale value and offset value for each type of attribute information. This allows the inverse transform unit 13912 to use different scale values ​​and offset values ​​between units or types.

[0449] Next, a fourth example of the transform unit and inverse transform unit will be described. Fig. 71 is a block diagram showing the configuration of a three-dimensional data encoding device 13900C according to the fourth example. Note that this figure mainly shows processing units related to encoding attribute information, and omits processing units related to encoding position information.

[0450] 71 differs from the three-dimensional data encoding device 13900 shown in Fig. 51 in that a conversion unit 13901C has a different configuration from the conversion unit 13901. Specifically, the conversion unit 13901C further includes a scale value generation unit 13908.

[0451] The scale value generation unit 13908 generates, from the scale value with decimal precision determined by the attribute information precision determination unit 13904, the value of the integer part of the scale value, the value of the decimal part of the scale value, and the decimal part offset value of the scale value, which represent the scale value.

[0452] The additional information encoding unit 13906 encodes, as additional information, the conversion information generated by the scale value generation unit 13908, which includes the integer part of the scale value, the decimal part of the scale value, and the decimal part offset value of the scale value. This additional information is included in a bitstream and sent to the three-dimensional data decoding device.

[0453] 72 is a block diagram showing the configuration of a three-dimensional data decoding device 13910C according to the fourth example. Note that the drawing mainly shows processing units related to decoding of attribute information, and omits processing units related to decoding of position information.

[0454] 72 differs from the three-dimensional data decoding device 13910 shown in FIG. 52 in that the function of the descaling unit 13915C included in the inverse conversion unit 13912C is different from that of the descaling unit 13915.

[0455] The additional information decoding unit 13914 decodes the encoded additional information included in the bitstream to generate the integer part of the scale value, the decimal part of the scale value, and the decimal part offset value of the scale value. The inverse scale unit 13915C calculates the scale value using the integer part of the scale value, the decimal part of the scale value, and the decimal part offset value of the scale value. The inverse scale unit 13915C generates attribute information by inversely scaling the decoded attribute information using the calculated scale value.

[0456] A specific example of processing by the conversion unit 13901C in Example 4 will be described below. For example, in an example where the three-dimensional data encoding device 13900C divides the attribute information by scale=255, and the three-dimensional data decoding device 13910C multiplies the decoded attribute information by scale=255, as shown in Figures 53 and 55, or 62 and 63, the scale value (scale) can be expressed as shown in the following (Equation a2) using scale_int, which is the integer part of the scale value, scale_frac, which is the decimal part of the scale value, and scale_frac_offset, which is the decimal part offset value of the scale value.

[0457]

number

[0458] Here, if scale_int=255, scale_frac=0, and scale_frac_offset=0, then scale=255.

[0459] Alternatively, the numerator and denominator in the above formula (a2) may be interchanged to define the scale. For example, if scale_int=1, scale_frac=8, and scale_frac_offset=1, then scale=1 / 255.

[0460] The scale value may be derived using the following formula a3: In this case, if scale_int=255 and scale_frac=0, then scale=255.

[0461]

number

[0462] In addition, the three-dimensional data encoding device 13900C and the three-dimensional data decoding device 13910C may always use either (Equation a2) or (Equation a3), or may selectively use either (Equation a2) or (Equation a3) by switching between them.

[0463] As a result, the three-dimensional data encoding device 13900C can encode attribute information with a bit depth of 16 bits as 8-bit attribute information, allowing it to handle smaller values ​​than when encoding 16-bit attribute information. This improves compression efficiency. Furthermore, the three-dimensional data encoding device 13900C can perform scaling that requires decimal point precision.

[0464] Furthermore, the three-dimensional data decoding device 13910C can decode the bit stream generated by the three-dimensional data encoding device 13900C and restore the original attribute information.

[0465] Furthermore, the inverse conversion unit 13912C included in the three-dimensional data decoding device 13910C performs inverse conversion processing on the decoded attribute information based on the conversion information included in the bitstream, thereby restoring the attribute information before it was converted by the conversion unit 13901C included in the three-dimensional data encoding device 13900C. Note that the three-dimensional data decoding device 13910C does not necessarily have to perform inverse conversion processing, and may select whether or not to perform inverse conversion processing based on the application or use case.

[0466] Note that here, an example has been described in which the three-dimensional data encoding device 13900C is equipped with an attribute information accuracy determination unit 13904 that determines the accuracy of the attribute information to be encoded, as in the first example, but the three-dimensional data encoding device 13900C may also be equipped with an attribute information resolution determination unit 13904A that determines the resolution or resolution of the attribute information, or an attribute information range determination unit 13904B that determines the range of the attribute information, as in the second or third example.

[0467] Furthermore, the conversion unit 13901C does not need to check the precision of the attribute information. For example, if the position of the significant bits of the attribute information is determined by the format standard of the attribute information to be encoded, the conversion unit 13901C may determine the scale value (scale) according to that rule.

[0468] Furthermore, even if the attribute information after scaling is in the 8-bit range, the conversion unit 13901C may output 16-bit attribute information with the upper 8 bits remaining invalid (filled with 0) as the converted attribute information without changing the bit depth to 8 bits.

[0469] Furthermore, although the above specific example has been described with respect to 16-bit attribute information, the same technique can be applied to attribute information of any bit depth.

[0470] The conversion unit 13901C may further include an offset unit that performs offset processing. In this case, the offset processing may be performed after or before the scaling processing.

[0471] Furthermore, the conversion unit 13901C may change the scale value for each coding group unit, such as a slice, a tile, or a frame. Furthermore, the conversion unit 13901C may change the scale value for each type of attribute information. This allows the conversion unit 13901C to use different scale values ​​between units or types.

[0472] Furthermore, the inverse conversion unit 13912C may further include an inverse offset unit that performs inverse offset processing. In this case, the inverse scaling processing may be performed after or before the inverse offset processing.

[0473] Furthermore, the inverse transform unit 13912C may perform inverse transform by changing the scale value for each coding group unit, such as a slice, a tile, or a frame. Furthermore, the inverse transform unit 13912C may change the scale value for each type of attribute information. This allows the inverse transform unit 13912C to use different scale values ​​between units or types.

[0474] Any two or more of the above first to fourth examples may be combined. Similarly to the attribute information, any one of the first to fourth examples or a combination of any two or more of them may be applied to the location information. In this case, the conversion information or format information may be stored in additional information such as an SPS.

[0475] A fifth example of the conversion unit and the inverse conversion unit will be described below. The conversion unit may include an offset calculation unit. This offset calculation unit calculates an offset component of the entire attribute information. The conversion unit may offset the attribute information using the calculated offset component (offset value). The offset calculation unit may calculate the offset component using attribute information of the entire sequence, or may calculate the offset component for each frame using attribute information in the frame.

[0476] Fig. 73 is a diagram showing an example of this conversion process. In the example shown in Fig. 73, the attribute information is 64-bit time information (time), and the conversion unit calculates an offset component for each frame. In Frame #1, one piece of time information (gps_time) is added to each point as attribute information, and the points are arranged in ascending order by time. The time may be the time when the point was acquired, or may be difference information relative to an arbitrary reference.

[0477] In the example shown in Figure 73, the conversion unit scales the time information of multiple points by 100000, then calculates the component 2312000 that is common to all points as an offset component, and offsets this offset component from the scaled time information.

[0478] This allows the attribute information to be converted into 8-bit positive information. By converting 64-bit information into 8-bit positive values ​​in this way, the amount of code can be reduced.

[0479] Note that the three-dimensional data encoding device may encode the difference value from the previous point if the value of the attribute information is a simple increase or a simple decrease. Also, if all difference values ​​are the same, the three-dimensional data encoding device may include the difference value in the additional information and transmit it, and may not encode the attribute information for each point. In other words, the three-dimensional data encoding device may not include the attribute information for each point in the bit stream.

[0480] An example of a method for storing transformation information will be described below. The transformation information, attribute_parameter(i), including the scale value and offset value described above may be stored in the SPS, or may be stored outside the SPS. For example, the transformation information may be stored in an APS (Attribute Parameter Set), which is a parameter set related to encoding of attribute information, or in other additional information such as a slice header. Alternatively, the transformation information may be stored in additional information such as SEI (Supplemental Enhancement Information).

[0481] FIG. 74 is a diagram illustrating an example of the syntax of an SPS and an SEI. When conversion information is stored in an SEI as shown in FIG. 74, the SEI includes sps_idx and attribute_idx. This allows the three-dimensional data decoding device to identify corresponding attribute information described in the SPS. Here, attribute_idx indicates that the SEI corresponds to the attribute information indicated at the attribute_idx-th position among multiple pieces of attribute information indicated in the numAttribute loop in the SPS. Therefore, the three-dimensional data decoding device can determine that the attribute information to be processed corresponding to the SEI corresponds to the attribute information indicated at the attribute_idx-th position among multiple pieces of attribute information indicated in the numAttribute loop in the SPS having the same sps_idx as the sps_idx included in the SEI.

[0482] If the SPS contains conversion information, the conversion information is applied to the attribute information values ​​of the entire sequence. If conversion information exists in the frame-level SEI (frame unit) provided for each frame, the conversion information may be applied to the attribute information values ​​of the frame.

[0483] Furthermore, when conversion information exists in both the SPS and the frame-level SEI, the conversion information indicated in the frame-level SEI may be applied. Alternatively, the conversion information indicated in both the SPS and the frame-level SEI may be applied. In other words, the frame-level SEI including the conversion information does not need to be provided for all frames, but may be provided for some frames.

[0484] When transformation information exists in both the SPS and the frame-level SEI, the amount of processing required for analyzing the transformation information in the 3D data decoding device can be reduced by applying the transformation information in the SPS, i.e., by prioritizing the SPS. Furthermore, the 3D data encoding device may store in the bitstream information specifying whether to prioritize the SPS or the frame-level SEI.

[0485] Furthermore, when transformation information exists in both the SPS and the frame-level SEI, the 3D data encoding device may set the transformation coefficients of the SPS to the average value of multiple transformation coefficients of multiple SEIs. This allows the 3D data decoding device to perform processing using the transformation information included in the SPS instead if the SEI is missing due to a communication error or the like. This also reduces the difference in processing results in this case.

[0486] The PS may have levels, such as a frame-level PS (Parameter Set), a sequence-level PS, and a PCC sequence-level PS including multiple sequences. In this example, the PCC sequence level is the highest level, the sequence level is the next highest level, and the frame level is the lowest. When multiple levels exist, the conversion information may be stored in the following manner.

[0487] Default conversion information is stored in a higher-level PS. The default conversion information is conversion information that is used when no conversion information is used in a lower-level PS, and is conversion information that may be used at multiple lower levels. Furthermore, if the conversion information used at a lower level differs from the conversion information included in the higher-level PS, the conversion information is stored in the lower-level PS.

[0488] Alternatively, the conversion information may not be stored in the higher-level PS, but may be stored in the lower-level PS.

[0489] In addition, information indicating whether the conversion information is stored in the lower PS, the higher PS, or both may be stored in either the lower PS or the higher PS, or both.

[0490] Furthermore, information included in a lower PS may be included in a higher PS, or if a lower PS and a higher PS overlap, either one may not be included in the bitstream.

[0491] Figure 75 is a diagram showing an example of how conversion information is stored. The example shown in (A) of Figure 75 is an example where conversion information exists in SPS but does not exist in SEI. Furthermore, val1 is the decoded value (decoded attribute information) of attribute information belonging to frame 1, and val2 is the decoded value (decoded attribute information) of attribute information belonging to frame 2. The conversion attribute information val1' and val2' can be derived as follows using the conversion information (offset@sps and scale@sps) of SPS.

[0492] val1'=val1×scale@sps+offset@sps val2'=val2×scale@sps+offset@sps 75B shows an example in which no transformation information exists in the SPS but does exist in the frame-level SEI. In this case, the post-transformation attribute information val1' and val2' can be derived as follows using the transformation information of SEI1 in frame 1 (offset@SEI1 and scale@SEI1) and the transformation information of SEI2 in frame 2 (offset@SEI2 and scale@SEI2):

[0493] val1'=val1×scale@SEI1+offset@SEI1 val2'=val2×scale@SEI2+offset@SEI2 75C shows a case where conversion information exists in both the SPS and the SEI. In this case, the converted attribute information val1' and val2' can be derived as follows using the conversion information in the SEI.

[0494] val1'=val1×scale@SEI1+offset@SEI1 val2'=val2×scale@SEI2+offset@SEI2 Alternatively, the converted attribute information val1' and val2' can be derived as follows using both the conversion information of the SPS and the conversion information of the SEI.

[0495] val1'=val1×scale@SPS×scale@SEI1+offset@SPS+offset@SEI1 val2'=val2×scale@SPS×scale@SEI2+offset@SPS+offset@SEI2 (D) in Figure 75 shows a case where conversion information exists in the SPS, and some frames have conversion information in the SEI, while others do not. In this example, sequence-level conversion information exists in the SPS, and the presence or absence of frame-level conversion information varies for each frame. In this case, for frames for which frame-level conversion information exists, the frame-level conversion information is applied to the value of the attribute information of that frame. Furthermore, for frames for which frame-level conversion information does not exist, the sequence-level conversion information is applied. For example, the post-conversion attribute information val1' for frame 1 for which frame-level SEI does not exist can be derived as follows using the conversion information in the SPS:

[0496] val1'=val1×scale@SPS+offset@SPS Furthermore, the post-conversion attribute information val2' of frame 2, in which the frame-level SEI exists, can be derived as follows using the conversion information of the SEI.

[0497] val2'=val2×scale@SEI2+offset@SEI2 Although an example of conversion and inverse conversion of attribute information is described here, similar techniques can also be used for conversion and inverse conversion of position information.

[0498] When the encoding unit and decoding unit for position information or attribute information are specified as supporting encoding in a predetermined format (e.g., a positive integer of a predetermined number of bits or less), and point cloud data configured in one or more predefined point cloud formats is encoded or decoded, the three-dimensional data encoding device notifies the three-dimensional data decoding device of the format information using the SEI, which is extension information. This allows the three-dimensional data decoding device to convert the decoded point cloud data into point cloud data in the point cloud format before encoding using this format information. Furthermore, even when encoding point cloud data in two or more different point cloud formats, the three-dimensional data encoding device stores format information for each point cloud format in the SEI, allowing the three-dimensional data decoding device to restore the point cloud data in each point cloud format.

[0499] For example, if the attribute information to be encoded is a normal vector, data in the range of -1.000 to 1.000, and three-dimensional data expressed by floating points, the attribute information is converted to a positive integer and the converted attribute information is inversely converted using the conversion unit 13901B and inverse conversion unit 13912 of the third example. This makes it possible to realize compression processing of point cloud data by the encoding unit 13902 and decoding unit 13911 that supports processing of positive integers of a predetermined number of bits or less.

[0500] Furthermore, when the attribute information to be encoded is composed of data of a predetermined bit length, but only a portion of the bits are valid bits, the first example of the conversion unit 13901 and inverse conversion unit 13912 can be used to realize compression processing of point cloud data using the encoding unit 13902 and decoding unit 13911.

[0501] Furthermore, when the attribute information to be encoded is data that is composed of multiples of a predetermined value, such as reflectance, where all values ​​of the attribute information are multiples of 255, by using the conversion unit 13901A and inverse conversion unit 13912 of the second example, compression processing of point cloud data using the encoding unit 13902 and decoding unit 13911 can be realized.

[0502] Furthermore, if the attribute information to be encoded can be expressed as a relative value to a reference value, such as a timestamp, compression processing of point cloud data using an encoding unit and a decoding unit can be realized by using the conversion unit and inverse conversion unit of variant example 5.

[0503] Furthermore, the three-dimensional data encoding device can reduce the amount of information by offsetting the reference value and encoding the difference from the reference value.

[0504] As described above, this embodiment is useful in applications that convert various point cloud data formats and encode and decode using general-purpose encoding and decoding methods. For example, in point cloud data for AR (Augmented Reality) and VR (Virtual Reality), multiple pieces of color information, normal vectors, and the like may be added to points as attribute information. Therefore, by using the above method, efficient decoding and encoding can be achieved. Furthermore, when the decoded point cloud data is presented using a presentation device such as a display device, the three-dimensional data decoding device may output the decoded point cloud data directly to the presentation device without performing inverse conversion.

[0505] In addition, in point cloud data acquired by LiDAR and 3D map data acquired based on LiDAR, reflectance or timestamps may be added to points. In this case, too, efficient encoding and decoding can be achieved by using the above method.

[0506] Although an example has been shown in which format information or conversion information is stored in additional information (various parameter sets, SEI, inventory, etc.) included in the bitstream, the method for storing the format information or conversion information is not limited to this. For example, the format information or conversion information may not be stored in the bitstream and may be sent to the three-dimensional data decoding device in another format. For example, the three-dimensional data encoding device may store the conversion information in data or a file in a data format defined separately from the bitstream by another standard, and send this data or file together with the bitstream to the three-dimensional data decoding device. In this case, the transmission means used for the bitstream and the data including the conversion information may be the same or different.

[0507] Furthermore, if the transformation information is predetermined and known, the three-dimensional data encoding device does not need to send the transformation information to the three-dimensional data decoding device, or the three-dimensional data encoding device may send an identifier instead of the transformation information to the three-dimensional data decoding device.

[0508] As described above, the three-dimensional data encoding device according to this embodiment performs the processing shown in Fig. 76. The three-dimensional data encoding device transforms attribute information of three-dimensional points in at least one frame among a plurality of frames constituting a sequence (S13921), and generates a bitstream by encoding the transformed attribute information (S13922). The bitstream further includes at least one first parameter of the transformation provided for the sequence (e.g., offset, scale@sps in Fig. 75), and at least one second parameter of the transformation provided for each of the at least one frame (e.g., offset, scale@SEI1, offset, scale@SEI2 in Fig. 75). In other words, the three-dimensional data encoding device stores the first parameter and the second parameter in the bitstream.

[0509] For example, the bitstream includes first control information for each sequence (e.g., SPS) and second control information for each frame (frame-level SEI). The first control information includes at least one first parameter, and the second control information includes at least one second parameter. That is, the three-dimensional data encoding device stores the first parameter in the first control information and stores the second parameter in the second control information.

[0510] This allows the three-dimensional data encoding device to selectively control the switching of transformation parameters both on a sequence-by-sequence basis and on a frame-by-frame basis, thereby enabling appropriate transformation processing and improving encoding efficiency.

[0511] The first control information including at least one first parameter and the second control information including at least one second parameter may be set for each property (component).

[0512] For example, in a three-dimensional data decoding device that decodes attribute information from a bitstream, if the bitstream contains a second parameter corresponding to the frame to be processed, the second parameter is used. In other words, the three-dimensional data encoding device stores in the bitstream the second parameter that is used in the conversion process of the attribute information of the frame to be processed.

[0513] The three-dimensional data encoding device may perform transformation on some of the frames, and not on the other frames. This allows the three-dimensional data encoding device to reduce the amount of code by performing transformation on frames where the amount of code would be large if transformation were not performed. Furthermore, the three-dimensional data encoding device can reduce the amount of processing and improve processing speed by not performing transformation on frames where the effect of reducing the amount of code is small. In this way, the three-dimensional data encoding device can improve compression efficiency while also improving processing speed.

[0514] For example, in a three-dimensional data decoding device, if the bitstream does not include the second parameter corresponding to the frame to be processed, the first parameter is used. In other words, the three-dimensional data encoding device stores the first parameter, which is the parameter used in the conversion process of the attribute information of the frame to be processed, in the bitstream.

[0515] For example, the conversion involves multiplying or dividing the attribute information by a first value (e.g., a scale value) and / or adding or subtracting a second value (e.g., an offset value), and each of the at least one first parameter and the at least one second parameter indicates at least one of the first value or the second value.

[0516] For example, the bitstream includes multiple types of information about three-dimensional points, including attribute information, and the bitstream further includes first information (e.g., is_property) indicating whether each of the multiple types of information is to be compressed. In other words, the three-dimensional data encoding device stores the multiple types of information about three-dimensional points and the first information in the bitstream.

[0517] For example, the multiple types of information include multiple types of attribute information of the three-dimensional points (for example, multiple pieces of color information and normal vectors, etc.) For example, the multiple types of information include position information of the three-dimensional points.

[0518] For example, the three-dimensional data encoding device may determine whether to convert or not convert attribute information depending on the type of attribute information. Note that the three-dimensional data encoding device may determine whether to convert or not convert attribute information depending on the data size of the attribute information. For example, the multiple types of attribute information include first attribute information and second attribute information, and the three-dimensional data encoding device converts the first attribute information but does not convert the second attribute information. This allows the three-dimensional data encoding device to adaptively switch between converting or not converting attribute information depending on the type of attribute information.

[0519] For example, the three-dimensional data encoding device determines whether to compress or not compress the attribute information depending on the type of the attribute information. Note that the three-dimensional data encoding device may also determine whether to compress or not compress the attribute information depending on the data size of the attribute information. For example, the multiple types of attribute information include first attribute information and second attribute information, and the three-dimensional data encoding device compresses the first attribute information and does not compress the second attribute information. This allows the three-dimensional data encoding device to adaptively switch between compressing and not compressing the attribute information depending on the type of attribute information.

[0520] For example, the bitstream includes first control information (e.g., SPS) for a sequence, and the first control information includes information indicating a list of multiple types of information, and the first information includes information (sps_id) that identifies the first control information and information (component_id) that indicates the number of the information to be compressed in the list.

[0521] For example, the bitstream further includes second information (for example, format_id) indicating the format type of the point cloud data including the attribute information. That is, the three-dimensional data encoding device stores the second information in the bitstream.

[0522] For example, the three-dimensional data encoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0523] Furthermore, the three-dimensional data decoding device according to this embodiment performs the processing shown in FIG. 77. The three-dimensional data decoding device generates decoded attribute information by decoding the bitstream (S13931), and generates attribute information of three-dimensional points of at least one frame among the multiple frames constituting the sequence by inversely transforming the decoded attribute information (S13932). The bitstream further includes at least one first parameter of inverse transform (e.g., offset, scale@sps in FIG. 75) provided for the sequence, and at least one second parameter of inverse transform (e.g., offset, scale@SEI1, offset, scale@SEI2 in FIG. 75) provided for each of the at least one frame. In other words, the three-dimensional data encoding device stores the first parameter and the second parameter in the bitstream. For example, the three-dimensional data decoding device performs inverse transform using at least one first parameter or at least one second parameter.

[0524] For example, the bitstream includes first control information for each sequence (e.g., SPS) and second control information for each frame (frame-level SEI). The first control information includes at least one first parameter, and the second control information includes at least one second parameter. That is, the three-dimensional data decoding device obtains the first parameter from the first control information and the second parameter from the second control information.

[0525] This allows the three-dimensional data decoding device to decode attribute information from a bitstream with improved coding efficiency.

[0526] For example, when the bit stream includes a second parameter corresponding to the frame to be processed, the three-dimensional data decoding device inversely converts the decoding attribute information of the frame to be processed using the second parameter.

[0527] For example, if the bitstream does not include the second parameter corresponding to the frame to be processed, the three-dimensional data decoding device inversely converts the decoding attribute information of the frame to be processed using the first parameter.

[0528] For example, in the inverse transformation, the decoded attribute information is multiplied or divided by a first value (e.g., a scale value) and / or added or subtracted by a second value (e.g., an offset value), and each of the at least one first parameter and the at least one second parameter indicates at least one of the first value or the second value.

[0529] For example, the bitstream includes multiple types of information about three-dimensional points, including attribute information, and the bitstream further includes first information (e.g., is_property) indicating whether each of the multiple types of information is to be compressed. For example, the three-dimensional data decoding device uses the first information to decode or obtain at least one of the multiple types of information about three-dimensional points from the bitstream.

[0530] For example, the multiple types of information include multiple types of attribute information of the three-dimensional points (for example, multiple pieces of color information and normal vectors, etc.) For example, the multiple types of information include position information of the three-dimensional points.

[0531] For example, the bitstream includes first control information (e.g., SPS) for a sequence, and the first control information includes information indicating a list of multiple types of information, and the first information includes information (sps_id) that identifies the first control information and information (component_id) that indicates the number of the information to be compressed in the list.

[0532] For example, the bitstream further includes second information (for example, format_id) indicating the format type of the point cloud data including the attribute information.

[0533] For example, the three-dimensional data decoding device includes a processor and a memory, and the processor performs the above processing using the memory.

[0534] Although the three-dimensional data encoding device and the three-dimensional data decoding device according to the embodiments of the present disclosure have been described above, the present disclosure is not limited to these embodiments.

[0535] Furthermore, each processing unit included in the three-dimensional data encoding device and three-dimensional data decoding device according to the above embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0536] Furthermore, the integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI fabrication, or reconfigurable processors, which allow the connections and settings of circuit cells within LSIs to be reconfigured, may also be used.

[0537] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0538] Furthermore, the present disclosure may be realized as a three-dimensional data encoding method or a three-dimensional data decoding method executed by a three-dimensional data encoding device, a three-dimensional data decoding device, or the like.

[0539] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0540] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and an order other than the above may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0541] While the three-dimensional data encoding device and three-dimensional data decoding device according to one or more aspects have been described above based on the embodiments, the present disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the present embodiments, and configurations constructed by combining components of different embodiments, may also be included within the scope of one or more aspects. [Industrial Applicability]

[0542] The present disclosure is applicable to a three-dimensional data encoding device and a three-dimensional data decoding device. [Explanation of symbols]

[0543] 2700 Location information encoder 2701, 2711 8-ary tree generation part 2702, 2712 Geometric information calculation section 2703, 2713 Encoding table selection section 2704 Entropy coding unit 2710 Location information decoding unit 2714 Entropy Decoding Unit 3140 Attribute information encoder 3141, 3151 LoD generation section 3142, 3152 Surroundings Search Unit 3143, 3153 Forecasting Department 3144 Prediction residual calculation unit 3145 Quantization section 3146 Arithmetic encoder 3147, 3155 Inverse quantization section 3148, 3156 Decryption value generation unit 3149, 3157 memory 3150 Attribute Information Decoding Unit 3154 Arithmetic Decoding Unit 4601 Three-dimensional data encoding system 4602 Three-dimensional data decoding system 4603 Sensor Terminal 4604 External connection part 4611 Point Cloud Data Generation System 4612 Presentation section 4613 Encoding section 4614 Multiplexer 4615 Input / output section 4616 Control Unit 4617 Sensor information acquisition unit 4618 Point Cloud Data Generation Unit 4621 Sensor Information Acquisition Unit 4622 Input / output section 4623 Demultiplexer 4624 Decoding Unit 4625 Presentation section 4626 User Interface 4627 Control Unit 4630 First Encoding Section 4631 Location information encoder 4632 Attribute information encoder 4633 Additional information coding unit 4634 Multiplexer 4640 First Decoding Unit 4641 Demultiplexer 4642 Location Information Decoding Unit 4643 Attribute Information Decoding Unit 4644 Additional information decoding unit 4650 Second Encoding Section 4651 Additional Information Generation Unit 4652 Position image generation unit 4653 Attribute Image Generation Unit 4654 Video Encoding Unit 4655 Additional information coding unit 4656 Multiplexer 4660 Second Decoding Unit 4661 Demultiplexer 4662 Video Decoding Unit 4663 Additional information decoding unit 4664 Location information generator 4665 Attribute information generation section 4801 Encoding section 4802 Multiplexer 6600 Attribute information encoder 6601 Sorting section 6602 Haar transform unit 6603 Quantization section 6604, 6612 Inverse quantization section 6605, 6613 Inverse Haar transform unit 6606, 6614 memory 6607 Arithmetic encoder 6610 Attribute Information Decoding Unit 6611 Arithmetic Decoding Unit 10600 3D data encoding device 10610, 10660 conversion unit 10611 Scale part 10612 Offset part 10620, 10670 encoder 10621 Attribute information encoder 10622 Additional information coding unit 10630 Three-dimensional data decoding device 10640, 10680 Decoding section 10641 Attribute Information Decoding Unit 10642 Additional information decoding unit 10650, 10690 reverse conversion unit 10651 Reverse offset part 10652 Reverse scale part 13900, 13900A, 13900B, 13900C 3D Data Encoding Device 13901, 13901A, 13901B, 13901C converter 13902 Encoding section 13903 Scale part 13904 Attribute information accuracy judgment unit 13904A Attribute information resolution determination unit 13904B Attribute information range determination unit 13905 Attribute information encoder 13906 Additional information coding unit 13907 Offset part 13908 Scale value generator 13910, 13910C Three-dimensional data decoding device 13911 Decryption Unit 13912, 13912C reverse conversion unit 13913 Attribute Information Decoding Unit 13914 Additional information decoding unit 13915, 13915C reverse scale part A100 Attribute information encoder A101 LoD attribute information encoder A102 Conversion attribute information encoding unit A110 Attribute information decoding unit A111 LoD attribute information decoding unit A112 Transformation attribute information decoding unit

Claims

1. calculating a plurality of transformed attribute values ​​by adding an offset to each of the plurality of pre-transformed attribute values ​​of the plurality of three-dimensional points, each of the plurality of transformed attribute values ​​being an unsigned integer that can be compressed by an encoder that conforms to a predetermined format for attribute values; transmitting the offset to the encoder; Three-dimensional data conversion method.

2. The offset is a fixed value regardless of the attribute values ​​before conversion.

2. The three-dimensional data conversion method according to claim 1.

3. The offset is the minimum value among the plurality of pre-conversion attribute values.

2. The three-dimensional data conversion method according to claim 1.

4. the plurality of transformed attribute values ​​are calculated by scaling the plurality of pre-transformed attribute values ​​by a scale factor; The scale factor is transmitted to the encoder.

2. The three-dimensional data conversion method according to claim 1.

5. The calculation of the plurality of post-conversion attribute values ​​includes applying a rounding process to the plurality of pre-conversion attribute values.

2. The three-dimensional data conversion method according to claim 1.

6. Each of the plurality of pre-conversion attribute values ​​is a signed integer.

2. The three-dimensional data conversion method according to claim 1.

7. the number of dimensions of each of the plurality of pre-conversion attribute values ​​is two or more; In the calculation of the plurality of transformed attribute values, the offset is added in common to the two or more dimensions.

2. The three-dimensional data conversion method according to claim 1.

8. obtaining a plurality of transformed attribute values ​​and offsets of a plurality of 3D points from a decoder, each of the plurality of transformed attribute values ​​being an unsigned integer decompressible by the decoder in accordance with a predetermined format of the attribute value; subtracting the offset from the transformed attribute values ​​to calculate a plurality of pre-transformed attribute values ​​for the three-dimensional points; Three-dimensional data inversion method.

9. The offset is a fixed value regardless of the attribute values ​​before conversion.

9. The three-dimensional data inverse transformation method according to claim 8.

10. The offset is the minimum value among the plurality of pre-conversion attribute values.

9. The three-dimensional data inverse transformation method according to claim 8.

11. the plurality of transformed attribute values ​​are calculated by inverse scaling the plurality of transformed attribute values ​​by a scale factor; The scale factor is obtained from the decoder.

9. The three-dimensional data inverse transformation method according to claim 8.

12. The plurality of pre-conversion attribute values ​​are signed integers.

9. The three-dimensional data inverse transformation method according to claim 8.

13. the number of dimensions of the attribute value after the transformation is two or more; In the calculation of the plurality of pre-conversion attribute values, the offset is subtracted in common from the two or more dimensions.

9. The three-dimensional data inverse transformation method according to claim 8.

14. a processor; a memory; The processor uses the memory to: calculating a plurality of transformed attribute values ​​by adding an offset to each of the plurality of pre-transformed attribute values ​​of the plurality of three-dimensional points, each of the plurality of transformed attribute values ​​being an unsigned integer that can be compressed by an encoder that conforms to a predetermined format for attribute values; transmitting the offset to the encoder; Three-dimensional data conversion device.

15. a processor; a memory; The processor uses the memory to: obtaining a plurality of transformed attribute values ​​and offsets of a plurality of 3D points from a decoder, each of the plurality of transformed attribute values ​​being an unsigned integer decompressible by the decoder in accordance with a predetermined format of the attribute value; subtracting the offset from the transformed attribute values ​​to calculate a plurality of pre-transformed attribute values ​​for the three-dimensional points; Three-dimensional data inverse conversion device.

Images