Three-dimensional data generation method, encoded three-dimensional data processing method, three-dimensional data generation device, and encoded three-dimensional data processing device

Abstract

The invention relates to a three-dimensional data generation method, an encoded three-dimensional data processing method, a three-dimensional data generation device, and an encoded three-dimensional data processing device. The three-dimensional data generation method comprises the following steps: generating a geometric information unit; generating an attribute information unit; and generating a unit including a plurality of parameters for decoding the geometric information unit and the attribute information unit, in which each of the geometric information unit, the attribute information unit, and the unit including the plurality of parameters includes information indicating a type of data included in the unit.

Description

[0001] This application is a divisional of the invention patent application with the application date of August 7, 2019, application number 201980051709.5, and invention title "3D Data Encoding Method, 3D Data Decoding Method, 3D Data Encoding Apparatus, and 3D Data Decoding Apparatus". Technical Field

[0002] The present disclosure relates to a 3D data encoding method, a 3D data decoding method, a 3D data encoding apparatus, and a 3D data decoding apparatus. Background Art

[0003] In large fields such as computer vision, map information, monitoring, infrastructure inspection, or video distribution for autonomous operation of automobiles or robots, devices or services that make flexible use of 3D data will be popularized in the future. 3D data is obtained by various methods such as distance sensors such as rangefinders, stereo cameras, or combinations of multiple monocular cameras.

[0004] As a representation method of 3D data, there is a representation method called point cloud, which represents the shape of a 3D structure by a point group in a 3D space. The position and color of the point group are stored in the point cloud. Although it is expected that the point cloud will become the mainstream as a representation method of 3D data, the data volume of the point group is very large. Therefore, in the storage or transmission of 3D data, like 2D moving images (as an example, MPEG-4 AVC or HEVC standardized as MPEG), data volume compression needs to be performed by encoding.

[0005] In addition, for the compression of point clouds, part of it is supported by publicly available libraries (PointCloud Library) that perform point cloud association processing.

[0006] In addition, there is a well-known technology that uses 3D map data to retrieve facilities around a vehicle and display them (for example, refer to Patent Document 1).

[0007] Prior Art Documents

[0008] Patent Documents

[0009] Patent Document 1: International Publication No. 2014 / 020663 Summary of the Invention

[0010] Problems to be Solved by the Invention

[0011] In the encoding process and decoding process of 3D data, it is desired to reduce the processing amount.

[0012] An object of the present disclosure is to provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding apparatus, or a three-dimensional data decoding apparatus capable of reducing the processing amount.

[0013] Means for Solving the Problem

[0014] A three-dimensional data generation method according to an aspect of the present disclosure includes: generating a geometric information unit; generating an attribute information unit; and generating a unit including a plurality of parameters for decoding the geometric information unit and the attribute information unit, wherein each of the geometric information unit, the attribute information unit, and the unit including the plurality of parameters includes information indicating the type of data included in the unit.

[0015] A method for processing encoded three-dimensional data according to an aspect of the present disclosure includes: obtaining a geometric information unit; obtaining an attribute information unit; and obtaining a unit including a plurality of parameters for decoding the geometric information unit, wherein each of the geometric information unit, the attribute information unit, and the unit including the plurality of parameters includes information indicating the type of data included in the unit.

[0016] Advantageous Effects of the Invention

[0017] The present disclosure can provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding apparatus, or a three-dimensional data decoding apparatus capable of reducing the processing amount. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a diagram showing the configuration of a three-dimensional data encoding / decoding system according to Embodiment 1.

[0019] Figure 2 is a diagram showing a structural example of point cloud data according to Embodiment 1.

[0020] Figure 3 is a diagram showing a structural example of a data file describing point cloud data information according to Embodiment 1.

[0021] Figure 4 is a diagram showing the types of point cloud data according to Embodiment 1.

[0022] Figure 5 is a diagram showing the configuration of the first encoding unit according to Embodiment 1.

[0023] Figure 6 is a block diagram of the first encoding unit according to Embodiment 1.

[0024] Figure 7 is a diagram showing the configuration of the first decoding unit according to Embodiment 1.

[0025] Figure 8It is a block diagram of the first decoding unit of Embodiment 1.

[0026] Figure 9 It is a diagram showing the structure of the second encoding unit of Embodiment 1.

[0027] Figure 10 It is a block diagram of the second encoding unit of Embodiment 1.

[0028] Figure 11 It is a diagram showing the structure of the second decoding unit of Embodiment 1.

[0029] Figure 12 It is a block diagram of the second decoding unit of Embodiment 1.

[0030] Figure 13 It is a diagram showing the protocol stack related to PCC encoded data in Embodiment 1.

[0031] Figure 14 It is a diagram showing the protocol stack of Embodiment 1.

[0032] Figure 15 It is a diagram showing the syntax example of the NAL unit in Embodiment 1.

[0033] Figure 16 It is a diagram showing the syntax example of the NAL unit header in Embodiment 1.

[0034] Figure 17 It is a diagram showing the semantic example of pcc_codec_type in Embodiment 1.

[0035] Figure 18 It is a diagram showing the semantic example of pcc_nal_unit_type in Embodiment 1.

[0036] Figure 19 It is a flowchart of the encoding process in Embodiment 1.

[0037] Figure 20 It is a flowchart of the decoding process of the second decoding unit in Embodiment 1.

[0038] Figure 21 It is a flowchart of the decoding process of the first decoding unit in Embodiment 1.

[0039] Figure 22 It is a diagram showing the protocol stack of Embodiment 2.

[0040] Figure 23 It is a diagram showing the syntax example of the NAL unit for codec 2 in Embodiment 2.

[0041] Figure 24 It is a diagram showing the syntax example of the NAL unit header for codec 2 in Embodiment 2.

[0042] Figure 25 It is a diagram showing a semantic example of codec2_nal_unit_type in Embodiment 2.

[0043] Figure 26 It is a diagram showing a syntax example of NAL units for Codec 1 in Embodiment 2.

[0044] Figure 27 It is a diagram showing a syntax example of the NAL unit header for Codec 1 in Embodiment 2.

[0045] Figure 28 It is a diagram showing a semantic example of codec1_nal_unit_type in Embodiment 2.

[0046] Figure 29 It is a flowchart of the encoding process in Embodiment 2.

[0047] Figure 30 It is a flowchart of the decoding process in Embodiment 2.

[0048] Figure 31 It is a diagram showing the protocol stack in Embodiment 3.

[0049] Figure 32 It is a diagram showing a syntax example of NAL units in Embodiment 3.

[0050] Figure 33 It is a diagram showing a syntax example of the NAL unit header in Embodiment 3.

[0051] Figure 34 It is a diagram showing a semantic example of pcc_nal_unit_type in Embodiment 3.

[0052] Figure 35 It is a flowchart of the encoding process in Embodiment 3.

[0053] Figure 36 It is a flowchart of the decoding process in Embodiment 3.

[0054] Figure 37 It is a flowchart of the encoding process of a modified example of the embodiment.

[0055] Figure 38 It is a flowchart of the decoding process of a modified example of the embodiment.

[0056] Figure 39 It is a block diagram of the encoding section in Embodiment 4.

[0057] Figure 40 It is a block diagram of the decoding section in Embodiment 4.

[0058] Figure 41 It is a flowchart of the encoding process of Embodiment 4.

[0059] Figure 42 It is a flowchart of the decoding process of Embodiment 4.

[0060] Figure 43 It is a diagram showing the basic structure of ISOBMFF related to Embodiment 5.

[0061] Figure 44 It is a diagram showing the protocol stack related to Embodiment 5.

[0062] Figure 45 It is a diagram showing an example of storing NAL units in a file for Codec 1 related to Embodiment 5.

[0063] Figure 46 It is a diagram showing an example of storing NAL units in a file for Codec 2 related to Embodiment 5.

[0064] Figure 47 It is a diagram showing the configuration of the first multiplexing section related to Embodiment 5.

[0065] Figure 48 It is a diagram showing the configuration of the first demultiplexing section related to Embodiment 5.

[0066] Figure 49 It is a diagram showing the configuration of the second multiplexing section related to Embodiment 5.

[0067] Figure 50 It is a diagram showing the configuration of the second demultiplexing section related to Embodiment 5.

[0068] Figure 51 It is a flowchart of the process performed by the first multiplexing section related to Embodiment 5.

[0069] Figure 52 It is a flowchart of the process performed by the second multiplexing section related to Embodiment 5.

[0070] Figure 53 It is a flowchart of the process performed by the first demultiplexing section and the first decoding section related to Embodiment 5.

[0071] Figure 54 It is a flowchart showing the process performed by the second demultiplexing section and the second decoding section related to Embodiment 5.

[0072] Figure 55 It is a diagram showing the configuration of the encoding section and the third multiplexing section related to Embodiment 6.

[0073] Figure 56It is a diagram showing the configuration of the third demultiplexing unit and decoding unit related to Embodiment 6.

[0074] Figure 57 It is a flowchart of the processing performed by the third multiplexing unit related to Embodiment 6.

[0075] Figure 58 It is a flowchart of the processing performed by the third demultiplexing unit and decoding unit related to Embodiment 6.

[0076] Figure 59 It is a flowchart of the processing performed by the three-dimensional data storage device related to Embodiment 6.

[0077] Figure 60 It is a flowchart of the processing performed by the three-dimensional data acquisition device related to Embodiment 6.

[0078] Figure 61 It is a diagram showing the configuration of the encoding unit and multiplexing unit related to Embodiment 7.

[0079] Figure 62 It is a diagram showing an example of the configuration of the encoded data related to Embodiment 7.

[0080] Figure 63 It is a diagram showing an example of the configuration of the encoded data and NAL unit related to Embodiment 7.

[0081] Figure 64 It is a diagram showing a semantic example of pcc_nal_unit_type related to Embodiment 7.

[0082] Figure 65 It is a diagram showing an example of the transmission order of the NAL unit related to Embodiment 7.

[0083] Figure 66 It is a flowchart of the processing performed by the three-dimensional data encoding device related to Embodiment 7.

[0084] Figure 67 It is a flowchart of the processing performed by the three-dimensional data decoding device related to Embodiment 7.

[0085] Figure 68 It is a flowchart of the multiplexing process related to Embodiment 7.

[0086] Figure 69 It is a flowchart of the demultiplexing process related to Embodiment 7.

[0087] Figure 70 It is a flowchart of the processing performed by the three-dimensional data encoding device related to Embodiment 7.

[0088] Figure 71It is a flowchart of the processing performed by the three-dimensional data decoding device according to Embodiment 7. Detailed Embodiment

[0089] A three-dimensional data encoding method according to an aspect of the present disclosure is a three-dimensional data encoding method for encoding time-series three-dimensional data, where the three-dimensional data includes position information and attribute information at each time. The three-dimensional data encoding method encodes the position information and encodes the attribute information of the processing object with reference to the position information at the same time as the attribute information of the processing object. The position information and the attribute information at the same time constitute an access unit.

[0090] Thereby, this three-dimensional data encoding method can facilitate the control of the reference in encoding using the access unit. Therefore, this three-dimensional data encoding method can reduce the processing amount of the encoding process.

[0091] For example, it may also be that the three-dimensional data encoding method further generates a bitstream including the encoded position information, the encoded attribute information, and the position information indicating the reference target of the attribute information of the processing object.

[0092] For example, it may also be that the bitstream includes: a position parameter set including control information of the position information at each time, and an attribute parameter set including control information of the attribute information at each time.

[0093] For example, it may also be that the bitstream includes: a position sequence parameter set including control information common to the position information at multiple times, and an attribute sequence parameter set including control information common to the attribute information at multiple times.

[0094] For example, it may also be that the bitstream includes: a stream parameter set including control information common to the position information at multiple times and the attribute information at multiple times.

[0095] For example, it may also be that the bitstream includes: an access unit header including control information common to the access unit.

[0096] For example, it may also be that encoding is performed in such a way that a group of pictures (GOF) composed of one or more of the access units can be decoded independently.

[0097] For example, it may also be that the bitstream includes: a GOF header including control information common to the GOF.

[0098] A three-dimensional data decoding method according to an aspect of the present disclosure is a three-dimensional data decoding method for decoding time-series three-dimensional data, the three-dimensional data including position information and attribute information at each time, and the position information and the attribute information at the same time constitute an access unit. The three-dimensional data decoding method decodes the position information from a bitstream, and decodes the attribute information of a processing object from the bitstream with reference to the position information at the same time as the attribute information of the processing object.

[0099] Thereby, the three-dimensional data decoding method can facilitate the control of the reference in decoding using the access unit. Therefore, the three-dimensional data decoding method can reduce the processing amount of the decoding process.

[0100] For example, it may also be that the three-dimensional data decoding method further obtains information representing the position information of a reference target of the attribute information of the processing object from the bitstream, and decodes the attribute information of the processing object with reference to the position information of the reference target represented by the obtained information.

[0101] For example, it may also be that the bitstream includes: a position parameter set of control information including position information at each time, and an attribute parameter set of control information including attribute information at each time.

[0102] For example, it may also be that the bitstream includes: a position sequence parameter set of control information common to the position information at a plurality of times, and an attribute sequence parameter set of control information common to the attribute information at a plurality of times.

[0103] For example, it may also be that the bitstream includes: a stream parameter set of control information common to the position information at a plurality of times and the attribute information at a plurality of times.

[0104] For example, it may also be that the bitstream includes: an access unit header including control information common to the access unit.

[0105] For example, it may also be that decoding is independently performed on a group of pictures (GOP) composed of one or more of the access units.

[0106] For example, it may also be that the bitstream includes: a GOP header including control information common to the GOP.

[0107] In addition, a three-dimensional data encoding device according to an aspect of the present disclosure is a three-dimensional data encoding device that encodes time-series three-dimensional data, and includes a processor and a memory. The three-dimensional data includes position information and attribute information at each time. The processor uses the memory to encode the position information, and encodes the attribute information of a processing object with reference to the position information at the same time as the attribute information of the processing object. The position information and the attribute information at the same time constitute an access unit.

[0108] Thereby, the three-dimensional data encoding device can facilitate the control of reference in encoding using the access unit. Therefore, the three-dimensional data encoding device can reduce the processing amount of the encoding process.

[0109] In addition, a three-dimensional data decoding device according to an aspect of the present disclosure is a three-dimensional data decoding device that decodes time-series three-dimensional data, and includes a processor and a memory. The three-dimensional data includes position information and attribute information at each time. The position information and the attribute information at the same time constitute an access unit. The processor uses the memory to decode the position information from the bitstream, and decodes the attribute information of the processing object from the bitstream with reference to the position information at the same time as the attribute information of the processing object.

[0110] Thereby, the three-dimensional data decoding device can facilitate the control of reference in decoding using the access unit. Therefore, the three-dimensional data decoding device can reduce the processing amount of the decoding process.

[0111] Furthermore, these general or specific aspects can be implemented by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and can be implemented by any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0112] Hereinafter, embodiments will be specifically described with reference to the drawings. In addition, all of the embodiments to be described below are specific examples showing the present disclosure. The numerical values, shapes, materials, constituent elements, arrangement positions and connection forms of the constituent elements, steps, and the order of steps shown in the following embodiments are all examples, and their gist is not to limit the present disclosure. In addition, among the constituent elements of the following embodiments, the constituent elements not described in the technical solution showing the uppermost concept are described as optional constituent elements.

[0113] (Embodiment 1)

[0114] When using the encoded data of the point cloud for an actual device or service, in order to suppress the network bandwidth, it is desired to transmit and receive the required information according to the usage. However, such a function does not exist in the encoding structure of three-dimensional data so far, and there is no corresponding encoding method.

[0115] In the present embodiment, a three-dimensional data encoding method, a three-dimensional data encoding device, a three-dimensional data decoding method, a three-dimensional data decoding device, a three-dimensional data multiplexing method for multiplexing the encoded data, and a three-dimensional data transmission method for transmitting the encoded data, which are used to provide a function of transmitting and receiving required information according to usage in the encoded data of three-dimensional point clouds, will be described.

[0116] In particular, currently, as encoding methods (encoding formats) for point cloud data, a first encoding method and a second encoding method have been studied, but a method for defining the structure of encoded data and saving the encoded data into a system format has not been defined. In this case, there are problems such as being unable to directly perform MUX processing (multiplexing) in the encoding unit, or transmission or storage.

[0117] In addition, a method for supporting a format in which two codecs, namely the first encoding method and the second encoding method, coexist, such as PCC (Point Cloud Compression), does not exist yet.

[0118] In the present embodiment, the structure of PCC encoded data in which two codecs, namely the first encoding method and the second encoding method, coexist and a method for saving the encoded data into a system format will be described.

[0119] First, the structure of the three-dimensional data (point cloud data) encoding and decoding system of the present embodiment will be described. Figure 1 is a diagram showing a structural example of the three-dimensional data encoding and decoding system of the present embodiment. As Figure 1 shown, the three-dimensional data encoding and decoding system includes a three-dimensional data encoding system 4601, a three-dimensional data decoding system 4602, a sensor terminal 4603, and an external connection unit 4604.

[0120] The three-dimensional data encoding system 4601 generates encoded data or multiplexed data by encoding point cloud data as three-dimensional data. In addition, the three-dimensional data encoding system 4601 may be a three-dimensional data encoding device implemented by a single device, or a system implemented by multiple devices. Further, the three-dimensional data encoding device may also be included in a part of the multiple processing units included in the three-dimensional data encoding system 4601.

[0121] The three-dimensional data encoding system 4601 includes a point cloud data generation system 4611, a hint unit 4612, an encoding unit 4613, a multiplexing unit 4614, an input / output unit 4615, and a control unit 4616. The point cloud data generation system 4611 includes a sensor information acquisition unit 4617 and a point cloud data generation unit 4618.

[0122] The sensor information acquisition unit 4617 acquires sensor information from the sensor terminal 4603 and outputs the sensor information to the point cloud data generation unit 4618. The point cloud data generation unit 4618 generates point cloud data based on the sensor information and outputs the point cloud data to the encoding unit 4613.

[0123] The prompting unit 4612 prompts the user with the sensor information or the point cloud data. For example, the prompting unit 4612 displays information or an image based on the sensor information or the point cloud data.

[0124] The encoding unit 4613 encodes (compresses) the point cloud data and outputs the obtained encoded data, control information obtained during the encoding process, and other additional information to the multiplexing unit 4614. The additional information includes, for example, the sensor information.

[0125] The multiplexing unit 4614 generates multiplexed data by multiplexing the encoded data, control information, and additional information input from the encoding unit 4613. The format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.

[0126] The input / output unit 4615 (e.g., a communication unit or an interface) outputs the multiplexed data to the outside. Alternatively, the multiplexed data is stored in a storage unit such as an internal memory. The control unit 4616 (or the application program execution unit) controls each processing unit. That is, the control unit 4616 performs control such as encoding and multiplexing.

[0127] In addition, the sensor information can be input to the encoding unit 4613 or the multiplexing unit 4614. Further, the input / output unit 4615 can also directly output the point cloud data or the encoded data to the outside.

[0128] The transmission signal (multiplexed data) output from the three-dimensional data encoding system 4601 is input to the three-dimensional data decoding system 4602 via the external connection unit 4604.

[0129] The three-dimensional data decoding system 4602 generates point cloud data as three-dimensional data by decoding the encoded data or the multiplexed data. In addition, the three-dimensional data decoding system 4602 can be a three-dimensional data decoding device implemented by a single device or a system implemented by multiple devices. Further, the three-dimensional data decoding device can also include a part of the multiple processing units included in the three-dimensional data decoding system 4602.

[0130] The three-dimensional data decoding system 4602 includes a sensor information acquisition unit 4621, an input / output unit 4622, a demultiplexing unit 4623, a decoding unit 4624, a prompting unit 4625, a user interface 4626, and a control unit 4627.

[0131] The sensor information acquisition unit 4621 acquires sensor information from the sensor terminal 4603.

[0132] The input / output unit 4622 acquires a transmission signal, decodes the multiplexed data (file format or packet) according to the transmission signal, and outputs the multiplexed data to the demultiplexing unit 4623.

[0133] The demultiplexing unit 4623 acquires encoded data, control information, and additional information from the multiplexed data, and outputs the encoded data, control information, and additional information to the decoding unit 4624.

[0134] The decoding unit 4624 reconstructs the point cloud data by decoding the encoded data.

[0135] The presentation unit 4625 presents the point cloud data to the user. For example, the presentation unit 4625 displays information or an image based on the point cloud data. The user interface 4626 acquires an instruction based on the user's operation. The control unit 4627 (or application execution unit) controls each processing unit. That is, the control unit 4627 controls demultiplexing, decoding, presentation, etc.

[0136] In addition, the input / output unit 4622 may also directly acquire point cloud data or encoded data from the outside. Additionally, the presentation unit 4625 may also acquire additional information such as sensor information, and present information based on the additional information. Additionally, the presentation unit 4625 may also perform presentation based on the instruction acquired by the user interface 4626 from the user.

[0137] The sensor terminal 4603 generates sensor information, which is the information obtained by the sensor. The sensor terminal 4603 is a terminal equipped with a sensor or a camera, such as a moving body like a car, a flying object like an airplane, a mobile terminal, or a camera.

[0138] The sensor information that can be acquired by the sensor terminal 4603 is, for example, (1) the distance between the sensor terminal 4603 and an object, or the reflectivity of the object, obtained by LIDAR, millimeter-wave radar, or an infrared sensor, (2) the distance between the camera and an object or the reflectivity of the object obtained from multiple monocular camera images or stereo camera images. Additionally, the sensor information may also include the posture, orientation, rotation (angular velocity), position (GPS information or altitude), speed, or acceleration of the sensor. Additionally, the sensor information may also include temperature, air pressure, humidity, or magnetism.

[0139] The external connection unit 4604 is implemented through an integrated circuit (LSI or IC), an external storage unit, communication with a cloud server via the Internet, or broadcasting, etc.

[0140] Next, the point cloud data will be described. Figure 2It is a diagram showing the structure of point cloud data. Figure 3 It is a diagram showing an example of the structure of a data file describing information on point cloud data.

[0141] Point cloud data contains data of multiple points. The data of each point contains position information (three-dimensional coordinates) and attribute information corresponding to this position information. A group that aggregates multiple such points is called a point cloud. For example, a point cloud represents the three-dimensional shape of an object (target).

[0142] Sometimes, position information (Position) such as three-dimensional coordinates is also called geometry. In addition, the data of each point can also contain attribute information (attribute) of multiple attribute categories. Attribute categories are, for example, color or reflectivity.

[0143] One piece of attribute information can be associated with one piece of position information, or attribute information with multiple different attribute categories can be associated with one piece of position information. In addition, multiple pieces of attribute information of the same attribute category can be associated with one piece of position information.

[0144] Figure 3 The example of the structure of the data file shown is an example of the case where position information and attribute information are in a one-to-one correspondence, representing the position information and attribute information of N points that make up the point cloud data.

[0145] Position information is, for example, information on the three axes of x, y, and z. Attribute information is, for example, RGB color information. As a representative data file, there is a ply file, etc.

[0146] Next, the types of point cloud data will be described. Figure 4 It is a diagram showing the types of point cloud data. As Figure 4 shown, point cloud data includes static objects and dynamic objects.

[0147] Static objects are three-dimensional point cloud data at any time (a certain moment). Dynamic objects are three-dimensional point cloud data that changes over time. Hereinafter, the three-dimensional point cloud data at a certain moment is called a PCC frame or a frame.

[0148] An object can be a point cloud with a certain degree of restricted area like ordinary image data, or a large-scale point cloud with an unrestricted area like map information.

[0149] In addition, there is point cloud data of various densities, and there can also be sparse point cloud data and dense point cloud data.

[0150] Hereinafter, the details of each processing unit will be described. Sensor information is obtained by various methods such as distance sensors like LIDAR or rangefinders, combinations of stereo cameras or multiple monocular cameras, etc. The point cloud data generation unit 4618 generates point cloud data based on the sensor information obtained by the sensor information acquisition unit 4617. The point cloud data generation unit 4618 generates position information as the point cloud data and attaches attribute information for this position information.

[0151] The point cloud data generation unit 4618 may also process the point cloud data when generating the position information or attaching the attribute information. For example, the point cloud data generation unit 4618 may reduce the data volume by deleting point clouds with duplicate positions. Additionally, the point cloud data generation unit 4618 may transform the position information (position transformation, rotation, or normalization, etc.), and may also render the attribute information.

[0152] In addition, in Figure 1 the point cloud data generation system 4611 is included in the three-dimensional data encoding system 4601, but it may also be independently set outside the three-dimensional data encoding system 4601.

[0153] The encoding unit 4613 encodes the point cloud data based on a pre-specified encoding method, thereby generating encoded data. There are roughly the following two types of encoding methods. The first type is an encoding method that uses the position information, and this encoding method will be described as the first encoding method hereafter. The second type is an encoding method that uses a video codec, and this storage method will be described as the second storage method hereafter.

[0154] The decoding unit 4624 decodes the encoded data based on a pre-specified encoding method, thereby decoding the point cloud data.

[0155] The multiplexing unit 4614 multiplexes the encoded data by using an existing multiplexing method, thereby generating multiplexed data. The generated multiplexed data is transmitted or stored. The multiplexing unit 4614 multiplexes other media such as images, sounds, subtitles, application programs, files, etc., or reference time information in addition to the PCC encoded data. Additionally, the multiplexing unit 4614 may also multiplex the attribute information associated with the sensor information or the point cloud data.

[0156] As the multiplexing method or file format, there are ISOBMFF, MPEG-DASH which is a transmission method based on ISOBMFF, MMT, MPEG-2TS Systems, RMP, etc.

[0157] The demultiplexing unit 4623 extracts the PCC encoded data, other media, and time information, etc. from the multiplexed data.

[0158] The input / output unit 4615 transmits the multiplexed data using a method consistent with a transmission medium such as broadcasting or communication or a storage medium. The input / output unit 4615 can communicate with other devices via the Internet or communicate with a storage unit such as a cloud server.

[0159] As a communication protocol, http, ftp, TCP, UDP, etc. are used. Either a PULL-type communication method or a PUSH-type communication method can be used.

[0160] Either wired transmission or wireless transmission can be used. As wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), or coaxial cable, etc. are used. As wireless transmission, wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), or millimeter wave, etc. are used.

[0161] In addition, as a broadcasting method, for example, DVB-T2, DVB-S2, DVB-C2, ATSC 3.0, or ISDB-S3, etc. are used.

[0162] Figure 5 It is a diagram showing the structure of the first encoding unit 4630, which is an example of the encoding unit 4613 that performs the first encoding method. Figure 6 It is a block diagram of the first encoding unit 4630. The first encoding unit 4630 generates encoded data (encoded stream) by encoding the point cloud data using the first encoding method. The first encoding unit 4630 includes a position information encoding unit 4631, an attribute information encoding unit 4632, an additional information encoding unit 4633, and a multiplexing unit 4634.

[0163] The first encoding unit 4630 is characterized by performing encoding while being aware of the three-dimensional structure. In addition, the first encoding unit 4630 is characterized in that the attribute information encoding unit 4632 performs encoding using the information obtained from the position information encoding unit 4631. The first encoding method is also referred to as GPCC (Geometry based PCC).

[0164] The point cloud data is PCC point cloud data such as a PLY file or PCC point cloud data generated based on sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData). The position information is input to the position information encoding unit 4631, the attribute information is input to the attribute information encoding unit 4632, and the additional information is input to the additional information encoding unit 4633.

[0165] The position information encoding unit 4631 generates encoded position information (Compressed Geometry) as encoded data by encoding the position information. For example, the position information encoding unit 4631 encodes the position information using an N-ary tree structure such as an octree. Specifically, in an octree, the object space is divided into 8 nodes (sub-spaces), and 8-bit information (occupancy encoding) indicating whether each node contains a point group is generated. In addition, the node containing the point group is further divided into 8 nodes, and 8-bit information indicating whether each of the 8 nodes contains a point group is generated. This process is repeated until it reaches a predetermined level or a threshold of the number of point groups contained in the node.

[0166] The attribute information encoding unit 4632 generates encoded attribute information (Compressed Attribute) as encoded data by encoding using the structure information generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 determines a reference point (reference node) to be referred to in the encoding of an object point (object node) of the object to be processed based on the octree structure generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 refers to a node in which the parent node in the octree of surrounding nodes or adjacent nodes is the same as the parent node of the object node. In addition, the method of determining the reference relationship is not limited to this.

[0167] In addition, the encoding process of the attribute information may include at least one of quantization processing, prediction processing, and arithmetic coding processing. In this case, reference means using the reference node in the calculation of the predicted value of the attribute information, or using the state of the reference node (for example, occupancy information indicating whether the reference node contains a point group) in the determination of the encoding parameter. For example, the encoding parameter is a quantization parameter in quantization processing or a context in arithmetic coding.

[0168] The additional information encoding unit 4633 generates encoded additional information (Compressed MetaData) as encoded data by encoding compressible data in the additional information.

[0169] The multiplexing unit 4634 generates an encoded stream (Compressed Stream) as encoded data by multiplexing the encoded position information, encoded attribute information, encoded additional information, and other additional information. The generated encoded stream is output to a processing unit in a system layer (not shown).

[0170] Next, a first decoding unit 4640, which is an example of a decoding unit 4624 that decodes using the first encoding method, will be described. Figure 7 It is a diagram showing the structure of the first decoding unit 4640. Figure 8It is a block diagram of the first decoding unit 4640. The first decoding unit 4640 generates point cloud data by decoding encoded data (encoded stream) encoded by the first encoding method using the first encoding method. The first decoding unit 4640 includes a demultiplexing unit 4641, a position information decoding unit 4642, an attribute information decoding unit 4643, and an additional information decoding unit 4644.

[0171] The encoded stream (Compressed Stream) as encoded data is input from a processing unit at the system layer (not shown) to the first decoding unit 4640.

[0172] The demultiplexing unit 4641 separates the encoded position information (Compressed Geometry), the encoded attribute information (Compressed Attribute), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0173] The position information decoding unit 4642 generates position information by decoding the encoded position information. For example, the position information decoding unit 4642 restores the position information of the point cloud represented by three-dimensional coordinates based on the encoded position information represented by an N-ary tree structure such as an octree.

[0174] The attribute information decoding unit 4643 decodes the encoded attribute information based on the structure information generated by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 determines a reference point (reference node) to be referred to in the decoding of an object point (object node) to be processed based on the octree structure obtained by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 refers to a node in which the parent node in the octree of a peripheral node or an adjacent node is the same as the parent node of the object node. In addition, the method of determining the reference relationship is not limited to this.

[0175] In addition, the decoding process of the attribute information may include at least one of an inverse quantization process, a prediction process, and an arithmetic decoding process. In this case, reference means using a reference node in the calculation of the predicted value of the attribute information, or using the state of the reference node in the determination of the decoded parameter (for example, indicating whether the occupancy information of the point cloud is included in the reference node). For example, the decoded parameter is a quantization parameter in the inverse quantization process, or a context in the arithmetic decoding.

[0176] The additional information decoding unit 4644 generates additional information by decoding the encoded additional information. In addition, the first decoding unit 4640 uses the additional information required for the decoding processes of the position information and the attribute information during decoding, and outputs the additional information required for the external application.

[0177] Next, a description will be given of a second encoding unit 4650, which is an example of an encoding unit 4613 that performs encoding using the second encoding method. Figure 9 This is a diagram showing the structure of the second encoding unit 4650. Figure 10 This is a block diagram of the second encoding unit 4650.

[0178] The second encoding unit 4650 generates encoded data (encoded stream) by encoding point cloud data using the second encoding method. The second encoding unit 4650 includes an additional information generation unit 4651, a position image generation unit 4652, an attribute image generation unit 4653, a video encoding unit 4654, an additional information encoding unit 4655, and a multiplexing unit 4656.

[0179] The second encoding unit 4650 has the following characteristics: it generates a position image and an attribute image by projecting a three-dimensional structure onto a two-dimensional image, and encodes the generated position image and attribute image using an existing video encoding method. The second encoding method is also referred to as VPCC (video based PCC, video-based PCC).

[0180] The point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated based on sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData).

[0181] The additional information generation unit 4651 generates mapping information of a plurality of two-dimensional images by projecting a three-dimensional structure onto a two-dimensional image.

[0182] The position image generation unit 4652 generates a position image (Geometry Image) based on the position information and the mapping information generated by the additional information generation unit 4651. This position image is, for example, a depth image representing distance as a pixel value. In addition, this depth image can be an image of observing a plurality of point clouds from one viewpoint (an image in which a plurality of point clouds are projected onto one two-dimensional plane), can be a plurality of images of observing a plurality of point clouds from a plurality of viewpoints, or can be one image formed by combining these plurality of images.

[0183] The attribute image generation unit 4653 generates an attribute image based on the attribute information and the mapping information generated by the additional information generation unit 4651. This attribute image is, for example, an image representing attribute information (such as color (RGB)) as a pixel value. In addition, this image can be an image of observing a plurality of point clouds from one viewpoint (an image in which a plurality of point clouds are projected onto one two-dimensional plane), can be a plurality of images of observing a plurality of point clouds from a plurality of viewpoints, or can be one image formed by combining these plurality of images.

[0184] The image encoding unit 4654 encodes the position image and the attribute image using an image encoding method, thereby generating a compressed geometry image and a compressed attribute image as encoded data. In addition, as the image encoding method, any known encoding method can be used. For example, the image encoding method is AVC or HEVC, etc.

[0185] The additional information encoding unit 4655 encodes the additional information, the mapping information, etc. included in the point cloud data to generate compressed metadata.

[0186] The multiplexing unit 4656 multiplexes the compressed geometry image, the compressed attribute image, the compressed metadata, and other additional information to generate a compressed stream as encoded data. The generated compressed stream is output to a processing unit in the system layer (not shown).

[0187] Next, the second decoding unit 4660, which is an example of the decoding unit 4624 that performs the second encoding method, will be described. Figure 11 It is a diagram showing the structure of the second decoding unit 4660. Figure 12 It is a block diagram of the second decoding unit 4660. The second decoding unit 4660 decodes the encoded data (compressed stream) encoded by the second encoding method using the second encoding method to generate point cloud data. The second decoding unit 4660 includes a demultiplexing unit 4661, an image decoding unit 4662, an additional information decoding unit 4663, a position information generation unit 4664, and an attribute information generation unit 4665.

[0188] The compressed stream as encoded data is input from a processing unit in the system layer (not shown) to the second decoding unit 4660.

[0189] The demultiplexing unit 4661 separates the compressed geometry image, the compressed attribute image, the compressed metadata, and other additional information from the encoded data.

[0190] The image decoding unit 4662 decodes the compressed geometry image and the compressed attribute image using an image encoding method to generate a position image and an attribute image. In addition, as the image encoding method, any known encoding method can be used. For example, the image encoding method is AVC or HEVC, etc.

[0191] The additional information decoding unit 4663 generates additional information including mapping information, etc. by decoding the encoded additional information.

[0192] The position information generation unit 4664 generates position information using the position image and mapping information. The attribute information generation unit 4665 generates attribute information using the attribute image and mapping information.

[0193] The second decoding unit 4660 uses the additional information required for decoding during decoding and outputs the additional information required for the external application.

[0194] Hereinafter, the problems in the PCC encoding method will be described. Figure 13 It is a diagram showing the protocol stack related to the PCC encoded data. Figure 13 It shows an example of multiplexing, transmitting, or storing data of other media such as images (e.g., HEVC) or sounds in the PCC encoded data.

[0195] The multiplexing method and file format have the function of multiplexing, transmitting, or storing various encoded data. In order to transmit or store the encoded data, the encoded data must be transformed into the format of the multiplexing method. For example, in HEVC, a technique is defined in which the encoded data is saved in a data structure called a NAL unit and the NAL unit is saved into ISOBMFF.

[0196] On the other hand, currently, as an encoding method for point cloud data, the first encoding method (Codec1) and the second encoding method (Codec2) are being studied, but the structure of the encoded data and the method of saving the encoded data into the system format are not defined, and there are problems such as being unable to directly perform the MUX processing (multiplexing), transmission, and storage in the encoding unit.

[0197] In addition, hereinafter, if there is no description of a specific encoding method, it means either the first encoding method or the second encoding method.

[0198] Hereinafter, the definition method of the NAL unit in the present embodiment will be described. For example, in conventional codecs such as HEVC, for one codec, one format of the NAL unit is defined. However, a method for supporting a format in which two codecs, the first encoding method and the second encoding method (hereinafter referred to as PCC codecs), coexist like PCC does not exist yet.

[0199] In the present embodiment, a format common to the PCC codecs is defined as the NAL unit, and further, an identifier of the NAL unit dependent on the PCC codecs is defined. Figure 14 It is a diagram showing the protocol stack in this case. Figures 15 to 17 It is a diagram showing an example of the NAL unit format common to the codecs.Figure 15 This is a diagram showing the syntax example of a Common PCC NAL Unit. Figure 16 This is a diagram showing the syntax example of a Common PCC NAL Unit Header. Figure 17 This is a diagram showing the semantic example of pcc_codec_type. Figure 18 This is a diagram showing an example of the definition of a codec-dependent NAL unit type and a semantic example of pcc_nal_unit_type.

[0200] As a NAL unit format, a NAL unit format common to the PCC codec is defined. A NAL unit (pcc_nal_unit) includes a header (pcc_nal_unit_header), a payload (pcc_nal_unit_payload), and trailing bits. The same format is used even when storing data of a codec that stores either the first encoding method or the second encoding method.

[0201] The codec type (pcc_codec_type) and the NAL unit type (pcc_nal_unit_type) are stored in the NAL unit header (pcc_nal_unit_header). The codec type indicates whether the PCC codec of the encoded data stored in the NAL unit is the first encoding method or the second encoding method.

[0202] The NAL unit type indicates the type of the codec-dependent NAL unit and is defined for each codec. When the codec type is the first encoding method, the NAL unit type indicates the NAL unit type defined for the first encoding method. When the codec type is the second encoding method, the NAL unit type indicates the NAL unit type defined for the second encoding method. That is, for the NAL unit type defined for the first encoding method and the NAL unit type defined for the second encoding method, the same value corresponds to different meanings.

[0203] In addition, in the header, the function of the codec type can also be incorporated into the NAL unit type. For example, part of the information of the NAL unit type can also be used to indicate the codec type.

[0204] Next, the encoding process of this embodiment will be described. Figure 19It is a flowchart of the encoding process of this embodiment. The process of this figure represents the process of the first encoding unit 4630 or the second encoding unit 4650 when the above definitions are used. In addition, hereinafter, the first encoding unit 4630 and the second encoding unit 4650 are not distinguished and are denoted as the encoding unit 4613. In addition, the process of this figure is mainly performed by Figure 6 the multiplexing unit 4634 shown or Figure 10 the multiplexing unit 4656 shown.

[0205] In addition, the process of this figure represents an example of encoding PCC data using either the first encoding method or the second encoding method, and it is known in which PCC codec the encoding is performed. For example, which PCC codec to use can also be specified by the user or an external device, etc.

[0206] First, the encoding unit 4613 encodes the PCC data using either the first encoding method or the second encoding method codec (S4601).

[0207] When the codec used is the second encoding method (the second encoding method in S4602), the encoding unit 4613 sets the pcc_codec_type included in the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4603). In addition, the encoding unit 4613 sets the identifier of the NAL unit for the second encoding method in the pcc_nal_unit_type of the NAL unit header (S4604). Then, the encoding unit 4613 generates a NAL unit that has the set NAL unit header and includes the encoded data in the payload. Then, the encoding unit 4613 transmits the generated NAL unit (S4605).

[0208] On the other hand, when the codec used is the first encoding method (the first encoding method in S4602), the encoding unit 4613 sets the pcc_codec_type of the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded by the first encoding method (S4606). In addition, the encoding unit 4613 sets the identifier of the NAL unit for the first encoding method in the pcc_nal_unit_type of the NAL unit header (S4607). Then, the encoding unit 4613 generates a NAL unit that has the set NAL unit header and includes the encoded data in the payload. Then, the encoding unit 4613 transmits the generated NAL unit (S4605).

[0209] In addition, in steps S4603 and S4606, when the function of pcc_code_type is included in pcc_nal_unit_type, the encoding unit 4613 may also indicate in pcc_nal_unit_type whether the NAL unit is encoded by the first encoding method or the second encoding method.

[0210] Next, the decoding processes of the first decoding unit 4640 and the second decoding unit 4660 of the present embodiment will be described. Figure 20 It is a flowchart showing the decoding process of the second decoding unit 4660. In addition, the process of this figure is mainly performed by Figure 12 the demultiplexing unit 4661 shown.

[0211] In addition, the process of this figure shows an example in which PCC data is encoded by either the second encoding method or the first encoding method. In addition, in this method, the demultiplexing unit 4661 included in the second decoding unit 4660 can identify the codec type of the NAL unit by referring to the information included in the NAL unit header. Therefore, the demultiplexing unit 4661 can output the required information to the video decoding unit 4662 according to the codec type.

[0212] First, the second decoding unit 4660 receives the NAL unit (S4611). For example, this NAL unit is generated in the process of the above-mentioned encoding unit 4613. That is, the header of this NAL unit includes pcc_codec_type and pcc_nal_unit_type.

[0213] Next, the second decoding unit 4660 determines whether pcc_codec_type included in the NAL unit header represents the first encoding method or the second encoding method (S4612).

[0214] When pcc_codec_type represents the second encoding method (the second encoding method in S4612), the second decoding unit 4660 determines that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4613). Then, the second decoding unit 4660 identifies the data by using pcc_nal_unit_type included in the NAL unit header as the identifier of the NAL unit for the second encoding method (S4614). Then, the second decoding unit 4660 decodes the PCC data using the decoding process of the second encoding method (S4615).

[0215] On the other hand, when pcc_codec_type indicates the first encoding method (the first encoding method in S4612), the second decoding unit 4660 determines that the data included in the payload of the NAL unit is the data encoded by the first encoding method (S4616). In this case, the second decoding unit 4660 does not process this NAL unit (S4617).

[0216] In addition, in step S4612, when the function of pcc_code_type is included in pcc_nal_unit_type, the second decoding unit 4660 can also refer to pcc_nal_unit_type to determine whether the codec used in the data included in the NAL unit is the first encoding method or the second encoding method.

[0217] Figure 21 It is a flowchart showing the decoding process of the first decoding unit 4640. In addition, the processing of this figure is mainly performed by Figure 8 the demultiplexing unit 4641 shown.

[0218] In addition, the processing of this figure shows an example in which PCC data is encoded by either the first encoding method or the second encoding method. In addition, in this method, the demultiplexing unit 4641 included in the first decoding unit 4640 can identify the codec type of the NAL unit by referring to the information included in the NAL unit header. Therefore, the demultiplexing unit 4641 can output the required information to the position information decoding unit 4642 and the attribute information decoding unit 4643 according to the codec type.

[0219] First, the first decoding unit 4640 receives the NAL unit (S4621). For example, this NAL unit is generated in the processing in the above-mentioned encoding unit 4613. That is, the header of this NAL unit includes pcc_codec_type and pcc_nal_unit_type.

[0220] Next, the first decoding unit 4640 determines whether pcc_codec_type included in the NAL unit header indicates the first encoding method or the second encoding method (S4622).

[0221] When pcc_codec_type indicates the second encoding method (the second encoding method in S4622), the first decoding unit 4640 determines that the data included in the payload of the NAL unit is the data encoded using the second encoding method (S4623). In this case, the first decoding unit 4640 does not process this NAL unit (S4624).

[0222] On the other hand, when pcc_codec_type indicates the first encoding method (the first encoding method in S4622), the first decoding unit 4640 determines that the data included in the payload of the NAL unit is the data encoded by the first encoding method (S4625). Then, the first decoding unit 4640 identifies the data by using pcc_nal_unit_type included in the NAL unit header as the identifier of the NAL unit for the first encoding method (S4626). Then, the first decoding unit 4640 decodes the PCC data by using the decoding process of the first encoding method (S4627).

[0223] (Embodiment 2)

[0224] In the present embodiment, another method for defining the NAL unit will be described. In the present embodiment, as the NAL unit, different formats are defined for each PCC codec. Furthermore, the identifier of the NAL unit is defined independently for each PCC codec.

[0225] Figure 22 It is a diagram showing the protocol stack in this case. Figure 23 It is a diagram showing a syntax example of the NAL unit (codec2_nal_unit) for codec 2. Figure 24 It is a diagram showing a syntax example of the NAL unit header (codec2_nal_unit_header) for codec 2. Figure 25 It is a diagram showing a semantic example of codec2_nal_unit_type.

[0226] Figure 26 It is a diagram showing a syntax example of the NAL unit (codec1_nal_unit) for codec 1. Figure 27 It is a diagram showing a syntax example of the NAL unit header (codec1_nal_unit_header) for codec 1. Figure 28 It is a diagram showing a semantic example of codec1_nal_unit_type.

[0227] As an NAL unit format, the NAL unit format is defined independently for each PCC codec. The NAL units (codec1_nal_unit, codec2_nal_unit) include headers (codec1_nal_unit_header, codec2_nal_unit_header), payloads (codec1_nal_unit_payload, codec2_nal_unit_payload), and trailing bits. The NAL units (codec1_nal_unit) used for the first encoding method and the NAL units (codec2_nal_unit) used for the second encoding method can have the same structure or different structures. The sizes of the NAL units used for the first encoding method and the NAL units used for the second encoding method can also be different.

[0228] The data encoded by the first encoding method is stored in the NAL units used for the first encoding method. The data encoded by the second encoding method is stored in the NAL units used for the second encoding method.

[0229] In the NAL unit headers (codec1_nal_unit_header, codec2_nal_unit_header), the NAL unit types (codec1_nal_unit_type, codec2_nal_unit_type) are stored. The NAL unit types are independent for each codec and are defined for each codec. That is, in the NAL units used for the first encoding method, the NAL unit types defined for the first encoding method are recorded. In the NAL units used for the second encoding method, the NAL unit types defined for the second encoding method are recorded.

[0230] By adopting this method, the first encoding method and the second encoding method can be processed as different codecs.

[0231] Next, the encoding process of this embodiment will be described. Figure 29 is a flowchart of the encoding process of this embodiment. The process in this figure represents the process of the first encoding unit 4630 or the second encoding unit 4650 when the above definitions are used. In addition, the process in this figure is mainly performed by Figure 6 the multiplexing unit 4634 shown in Figure 10 or the multiplexing unit 4656 shown in

[0232] In addition, the processing of this figure represents an example in which PCC data is encoded using either the first encoding method or the second encoding method, and it is known in which PCC codec the encoding is performed. For example, which PCC codec to use can also be specified by the user or an external device, etc.

[0233] First, the encoding unit 4613 encodes the PCC data using either one of the first encoding method and the second encoding method (S4631).

[0234] When the codec used is the second encoding method (the second encoding method in S4632), the encoding unit 4613 generates a NAL unit in the NAL unit format for the second encoding method (S4633). Next, the encoding unit 4613 sets the identifier of the NAL unit for the second encoding method in the codec2_nal_unit_type in the NAL unit header (S4634). Then, the encoding unit 4613 generates a NAL unit that has the set NAL unit header and includes the encoded data in the payload. Then, the encoding unit 4613 transmits the generated NAL unit (S4635).

[0235] On the other hand, when the codec used is the first encoding method (the first encoding method in S4632), the encoding unit 4613 generates a NAL unit in the NAL unit format for the first encoding method (S4636). Next, the encoding unit 4613 sets the identifier of the NAL unit for the first encoding method in the codec1_nal_unit_type in the NAL unit header (S4637). Then, the encoding unit 4613 generates a NAL unit that has the set NAL unit header and includes the encoded data in the payload. Then, the encoding unit 4613 transmits the generated NAL unit (S4635).

[0236] Next, the decoding process of this embodiment will be described. Figure 30 is a flowchart of the decoding process of this embodiment. The processing of this figure represents the processing using the first decoding unit 4640 or the second decoding unit 4660 in the case of the above definition. In addition, hereinafter, the first decoding unit 4640 or the second decoding unit 4660 will not be distinguished and will be denoted as the decoding unit 4624. In addition, the processing of this figure is mainly performed by Figure 8 the demultiplexing unit 4641 shown or Figure 12 the demultiplexing unit 4661 shown.

[0237] In addition, the processing of this figure represents an example in which PCC data is encoded using either the first encoding method or the second encoding method, and it is assumed that it is known in which PCC codec the encoding was performed. For example, information indicating the codec used is included in the transmission signal, multiplexed data, or encoded data, and the decoding unit 4624 determines the codec used by referring to this information. In addition, the decoding unit 4624 may also determine the codec used based on a signal obtained separately from these signals.

[0238] When the codec used is the second encoding method (second encoding method in S4641), the decoding unit 4624 receives NAL units in the format for the second encoding method (S4642). Then, assuming that the NAL unit is for the second encoding method, the decoding unit 4624 uses the NAL unit format for the second encoding method and the codec2_nal_unit_type for the second encoding method to identify the data (S4643). Then, the decoding unit 4624 decodes the PCC data using the decoding process of the second encoding method (S4644).

[0239] On the other hand, when the codec used is the first encoding method (first encoding method in S4641), the decoding unit 4624 receives NAL units in the format for the first encoding method (S4645). Then, assuming that the NAL unit is for the first encoding method, the decoding unit 4624 uses the NAL unit format for the first encoding method and the codec1_nal_unit_type for the first encoding method to identify the data (S4646). Then, the decoding unit 4624 decodes the PCC data using the decoding process of the first encoding method (S4747).

[0240] (Embodiment 3)

[0241] In this embodiment, another method for defining NAL units will be described. In this embodiment, a format common to PCC codecs is defined as the NAL unit. In addition, an identifier for the NAL unit common to PCC codecs is defined.

[0242] Figure 31 It is a diagram showing the protocol stack in this case. Figures 32 to 34 It is a diagram showing an example of the NAL unit format common to codecs. Figure 32 It is a diagram showing a syntax example of a Common PCC NAL Unit. Figure 33 It is a diagram showing a syntax example of a Common PCC NAL Unit Header. Figure 34 It is a diagram showing a semantic example of pcc_codec_type.

[0243] As a NAL unit format, a common NAL unit format for the PCC codec is defined. The NAL unit (pcc_nal_unit) includes a header (pcc_nal_unit_header), a payload (pcc_nal_unit_payload), and trailing bits. The same format is also used when storing data of any one of the first encoding method and the second encoding method.

[0244] In the NAL unit header (pcc_nal_unit_header), the NAL unit type (pcc_nal_unit_type) is stored. The NAL unit type is common to the codecs and defines a common type for the codecs. That is, the NAL unit used for the first encoding method and the NAL unit used for the second encoding method together record the commonly defined NAL unit type. In Figure 34 the example shown, for example, PCC DataA is the encoded data of codec 1, PCC DataB is the encoded data of codec 2, PCCMetaDataA is the additional information of codec 1, and PCC MetaDataB is the additional information of codec 2.

[0245] By adopting this method, the first encoding method and the second encoding method can be treated as the same codec.

[0246] Next, the encoding process of this embodiment will be described. Figure 35 is a flowchart of the encoding process of this embodiment. The process in this figure represents the process of the first encoding unit 4630 or the second encoding unit 4650 when the above definition is used. In addition, the process in this figure is mainly performed by Figure 6 the multiplexing unit 4634 shown or Figure 10 the multiplexing unit 4656 shown.

[0247] In addition, the process in this figure represents an example of encoding PCC data using either the second encoding method or the first encoding method, and it is known in which PCC codec the encoding is performed. For example, which PCC codec to use can also be specified by the user or an external device, etc.

[0248] First, the encoding unit 4613 encodes the PCC data using any one of the second encoding method and the first encoding method (S4651). Next, the encoding unit 4613 generates a NAL unit in the PCC common NAL unit format (S4652).

[0249] Next, the encoding unit 4613 sets an identifier of a PCC common NAL unit for pcc_nal_unit_type included in the NAL unit header (S4653). Next, it transmits an NAL unit having the set NAL unit header and including encoded data in the payload (S4654).

[0250] Next, the decoding process of this embodiment will be described. Figure 36 It is a flowchart of the decoding process of this embodiment. The process in this figure represents the process of the first decoding unit 4640 or the second decoding unit 4660 when the above definitions are used. In addition, the process in this figure is mainly performed by Figure 8 the demultiplexing unit 4641 shown or Figure 12 the demultiplexing unit 4661 shown.

[0251] In addition, the process in this figure represents an example in which PCC data is encoded by either the second encoding method or the first encoding method.

[0252] First, the decoding unit 4624 determines the codec used in the encoding of the data included in the NAL unit (S4661). For example, the decoding unit 4624 determines the codec used by referring to pcc_nal_unit_type included in the NAL unit header.

[0253] When the codec used is the second encoding method (the second encoding method in S4661), the decoding unit 4624 receives an NAL unit in the PCC common format (S4662). Next, assuming that the NAL unit is common, the decoding unit 4624 identifies the data using the common NAL unit format and the common pcc_nal_unit_type (S4663). Next, the decoding unit 4624 decodes the PCC data using the decoding process of the second encoding method (S4664).

[0254] On the other hand, when the codec used is the first encoding method (the first encoding method in S4661), the decoding unit 4624 receives an NAL unit in the PCC common format (S4665). Next, assuming that the NAL unit is common, the decoding unit 4624 identifies the data using the common NAL unit format and the common pcc_nal_unit_type (S4666). Next, the decoding unit 4624 decodes the PCC data using the decoding process of the first encoding method (S4667).

[0255] Hereinafter, modification examples of the above-described Embodiment 1 to Embodiment 3 will be described. As another method of indicating the PCC codec type, the following method can also be used.

[0256] In Embodiment 1, Embodiment 2, and Embodiment 3, the case where two codecs, i.e., the first coding method and the second coding method, coexist has been described. However, the above-described method can also be applied when there are three or more PCC codecs.

[0257] In addition, in Embodiment 1 and Embodiment 3, the identification information of the PCC codec (pcc_codec_type in Embodiment 1 and pcc_nal_unit_type in Embodiment 3b) is described in the NAL unit header. However, the identification information of the codec can also be stored in other places.

[0258] In addition, the first coding method and the second coding method are not limited to the above examples and can be any codecs. For example, the first coding method and the second coding method can be multiple codecs obtained by subdividing GPCC, or can be multiple codecs obtained by subdividing VPCC. For example, it can also be the case where both the first coding method and the second coding method are VPCC, but different video coding methods are used. The video coding method can be, for example, AVC or HEVC. In addition, either or both of the first coding method and the second coding method can be a coding method including other coding methods such as video, audio, and text applications.

[0259] For example, the identification information of the codec can also be included in the control information included in the PCC coding stream. Here, the control information is, for example, metadata such as a parameter set or SEI (Supplemental Enhancement Information).

[0260] Figure 37 This is a flowchart of the encoding process of the encoding unit 4613 in this case. First, the encoding unit 4613 encodes the PCC data (S4671), and records the identification information of the PCC codec at a specified position (for example, a parameter set) in the encoded data (S4672). Then, the encoding unit 4613 generates a NAL unit including the encoded data, and transmits the generated NAL unit (S4673).

[0261] Alternatively, the identification information of the PCC codec can be defined as a profile, and the identification information of the PCC codec can be represented in the metadata. Additionally, in the case where the same codec is used for the entire sequence, the identification information of the PCC codec can also be included in the sequence parameter set. Furthermore, in the case where each PCC frame is encoded with a different codec, the identification information of the PCC codec can also be included in the parameter set that records the information of each frame. For example, in the case where different codecs are used for each piece of PCC data, such as when the codec is different in the position information and the attribute information, the identification information of the PCC codec can also be included in the parameter set that records the information of each piece of data. That is, the information of the codec representing the position information is included in the control information (parameter set, etc.) of the position information, and the information of the codec representing the attribute information can also be included in the control information (parameter set, etc.) of the attribute information.

[0262] In addition, the identification information of the codec can be stored in any one of the above, or can be stored in multiple locations. For example, the identification information of the codec can also be stored in both the encoded stream and the NAL unit header. Additionally, in the case where the identification information of the codec is stored in multiple locations, the same information or different information can be stored in the multiple locations. Different information is, for example, information indicating GPCC or VPCC, and information indicating any one of the multiple codecs obtained by subdividing GPCC or VPCC.

[0263] When the demultiplexing unit 4641 or 4661 included in the decoding unit 4624 includes a parameter set in the NAL unit, by analyzing the record in the parameter set, it is possible to determine whether the data included in the payload of the NAL unit is data encoded by the first encoding method or data encoded by the second encoding method. Thus, the decoding unit 4624 can quickly filter out NAL units that are not required for decoding.

[0264] Figure 38 It is a flowchart of the decoding process of the decoding unit 4624 in this case. First, the decoding unit 4624 receives the NAL unit (S4675), and uses the pcc_nal_unit_type included in the NAL unit header to identify the specified data (such as the above parameter set) that records the identification information of the PCC codec (S4676). Next, the decoding unit 4624 identifies the PCC codec indicated in the specified data (such as the parameter set) by analyzing the specified data (S4677). Next, the decoding unit 4624 decodes the encoded data using the identified PCC codec (S4678).

[0265] In addition, in the above, an example of storing an encoded stream in NAL units is shown, but a unit (cell) in a predetermined manner may be used instead of NAL units.

[0266] (Embodiment 4)

[0267] In this embodiment, an encoding unit 4670 having the functions of both the above-described first encoding unit 4630 and second encoding unit 4650, and a decoding unit 4680 having the functions of both the first decoding unit 4640 and second decoding unit 4660 will be described.

[0268] Figure 39 is a block diagram of the encoding unit 4670 of this embodiment. The encoding unit 4670 includes the above-described first encoding unit 4630, second encoding unit 4650, and a multiplexing unit 4671. The multiplexing unit 4671 multiplexes the encoded data generated by the first encoding unit 4630 and the encoded data generated by the second encoding unit 4650, and outputs the resulting encoded data.

[0269] Figure 40 is a block diagram of the decoding unit 4680 of this embodiment. The decoding unit 4680 includes the above-described first decoding unit 4640, second decoding unit 4660, and a demultiplexing unit 4681. The demultiplexing unit 4681 extracts the encoded data using the first encoding method and the encoded data using the second encoding method from the input encoded data. The demultiplexing unit 4681 outputs the encoded data using the first encoding method to the first decoding unit 4640, and outputs the encoded data using the second encoding method to the second decoding unit 4660.

[0270] According to the above structure, the encoding unit 4670 can selectively use the first encoding method and the second encoding method to encode point cloud data. In addition, the decoding unit 4680 can decode the encoded data encoded using the first encoding method, the encoded data encoded using the second encoding method, and the encoded data encoded using both the first encoding method and the second encoding method.

[0271] For example, the encoding unit 4670 can switch the encoding methods (the first encoding method and the second encoding method) in units of point cloud data or frames. In addition, the encoding unit 4670 can also switch the encoding method in units that can be encoded.

[0272] The encoding unit 4670 generates, for example, encoded data (encoded stream) including the identification information of the PCC codec described in the above Embodiment 1 or Embodiment 3.

[0273] The demultiplexing unit 4681 included in the decoding unit 4680 identifies data using, for example, the identification information of the PCC codec described in Embodiment 1 or Embodiment 3. When the data is data encoded by the first encoding method, the demultiplexing unit 4681 outputs the data to the first decoding unit 4640, and when the data is data encoded by the second encoding method, the demultiplexing unit 4681 outputs the data to the second decoding unit 4660.

[0274] In addition, the encoding unit 4670 may send, as control information, information indicating whether both encoding methods or only one of the encoding methods is used, in addition to the identification information of the PCC codec.

[0275] Next, the encoding process of this embodiment will be described. Figure 41 is a flowchart of the encoding process of this embodiment. By using the identification information of the PCC codec described in Embodiment 1, Embodiment 2, Embodiment 3, and the modification example, an encoding process corresponding to multiple codecs can be performed. In addition, Figure 41 An example showing the case of using the method of Embodiment 1 is presented, but the same process can be applied to other methods.

[0276] First, the encoding unit 4670 encodes the PCC data using one or both of the first encoding method and the second encoding method by a codec (S4681).

[0277] When the codec used is the second encoding method (the second encoding method in S4682), the encoding unit 4670 sets the pcc_codec_type included in the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4683). Next, the encoding unit 4670 sets the identifier of the NAL unit for the second encoding method in the pcc_nal_unit_type of the NAL unit header (S4684). Then, the encoding unit 4670 generates a NAL unit having the set NAL unit header and including the encoded data in the payload. Then, the encoding unit 4670 transmits the generated NAL unit (S4685).

[0278] On the other hand, when the codec used is the first encoding method (the first encoding method in S4682), the encoding unit 4670 sets pcc_codec_type included in the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded by the first encoding method (S4686). Next, the encoding unit 4670 sets the identifier of the NAL unit for the first encoding method to pcc_nal_unit_type included in the NAL unit header (S4687). Next, the encoding unit 4670 generates a NAL unit having the set NAL unit header and including encoded data in the payload. Then, the encoding unit 4670 transmits the generated NAL unit (S4685).

[0279] Next, the decoding process of the present embodiment will be described. Figure 42 FIG. is a flowchart of the decoding process of the present embodiment. By using the identification information of the PCC codec described in Embodiment 1, Embodiment 2, Embodiment 3, and the modification example, decoding processes corresponding to a plurality of codecs can be performed. In addition, Figure 42 An example showing the case where the method of Embodiment 1 is used is shown, but the same process can be applied to other methods.

[0280] First, the decoding unit 4680 receives a NAL unit (S4691). For example, this NAL unit is generated in the process in the above-described encoding unit 4670.

[0281] Next, the decoding unit 4680 determines whether pcc_codec_type included in the NAL unit header indicates the first encoding method or the second encoding method (S4692).

[0282] When pcc_codec_type indicates the second encoding method (the second encoding method in S4692), the decoding unit 4680 determines that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4693). Then, the second decoding unit 4660 identifies the data by using pcc_nal_unit_type included in the NAL unit header as the identifier of the NAL unit for the second encoding method (S4694). Then, the decoding unit 4680 decodes the PCC data by using the decoding process of the second encoding method (S4695).

[0283] On the other hand, when pcc_codec_type indicates the first encoding method (the first encoding method in S4692), the decoding unit 4680 determines that the data included in the payload of the NAL unit is the data encoded by the first encoding method (S4696). Then, the decoding unit 4680 identifies the data using pcc_nal_unit_type included in the NAL unit header as the identifier of the NAL unit for the first encoding method (S4697). Then, the decoding unit 4680 decodes the PCC data using the decoding process of the first encoding method (S4698).

[0284] As described above, a three-dimensional data encoding device according to an aspect of the present disclosure generates an encoded stream by encoding three-dimensional data (e.g., point cloud data) (e.g., Figure 37 of S4671), and stores information indicating the encoding method used for the encoding among the first encoding method and the second encoding method (e.g., identification information of the codec) (e.g., Figure 37 of S4672) in the control information (e.g., parameter set) of the encoded stream.

[0285] Thereby, when decoding the encoded stream generated by this three-dimensional data encoding device, the three-dimensional data decoding device can use the information stored in the control information to determine the encoding method used in the encoding. Therefore, even when using multiple encoding methods, the three-dimensional data decoding device can correctly decode the encoded stream.

[0286] For example, the three-dimensional data includes position information. In the encoding, the three-dimensional data encoding device encodes the position information. In the storing, the three-dimensional data encoding device stores information indicating the encoding method used for the encoding of the position information among the first encoding method and the second encoding method in the control information of the position information.

[0287] For example, the three-dimensional data includes position information and attribute information. In the encoding, the three-dimensional data encoding device encodes the position information and the attribute information. In the storing, the three-dimensional data encoding device stores, in the control information of the position information, information indicating the encoding method used for the encoding of the position information among the first encoding method and the second encoding method, and stores, in the control information of the attribute information, information indicating the encoding method used for the encoding of the attribute information among the first encoding method and the second encoding method.

[0288] Thereby, different encoding methods can be used for the position information and the attribute information, so that the encoding efficiency can be improved.

[0289] For example, the three-dimensional data encoding method further stores the encoded stream in one or more units (e.g., NAL units) (e.g., Figure 37 of S4673).

[0290] For example, as described in Embodiment 1 Figures 15 to 18 the unit contains information that has a format common to the first encoding method and the second encoding method, the information is information indicating the category of data contained in the unit, and has a definition independent of the first encoding method and the second encoding method (e.g., pcc_nal_unit_type).

[0291] For example, as described in Embodiment 2 Figures 23 to 28 the unit contains information that has a format independent of the first encoding method and the second encoding method, the information is information indicating the category of data contained in the unit, and has a definition independent of the first encoding method and the second encoding method (e.g., codec1_nal_unit_type or codec2_nal_unit_type).

[0292] For example, as described in Embodiment 3 Figures 32 to 34 the unit contains information that has a format common to the first encoding method and the second encoding method, the information is information indicating the category of data contained in the unit, and has a definition common to the first encoding method and the second encoding method (e.g., pcc_nal_unit_type).

[0293] For example, a three-dimensional data encoding device includes a processor and a memory, and the processor uses the memory to perform the above processing.

[0294] In addition, the three-dimensional data decoding device of the present embodiment determines the encoding method used in the encoding of the encoded stream (e.g., Figure 38 of S4677) based on information indicating the encoding method for encoding the three-dimensional data in the first encoding method and the second encoding method (e.g., codec identification information) included in the control information (e.g., parameter set) of the encoded stream generated by encoding the three-dimensional data, and decodes the encoded stream using the determined encoding method (e.g., Figure 38 of S4678).

[0295] Thus, when decoding the encoded stream, the three-dimensional data decoding device can use the information stored in the control information to determine the encoding method used in the encoding. Therefore, even when multiple encoding methods are used, the three-dimensional data decoding device can correctly decode the encoded stream.

[0296] For example, the three-dimensional data includes position information, and the encoded stream includes encoded data of the position information. In the determination, the three-dimensional data decoding device determines the encoding method used in the encoding of the position information based on the information indicating the encoding method used in the encoding of the position information in the control information included in the encoded stream and included in the first encoding method and the second encoding method. In the decoding, the three-dimensional data decoding device decodes the encoded data of the position information using the encoding method determined to be used in the encoding of the position information.

[0297] For example, the three-dimensional data includes position information and attribute information, and the encoded stream includes encoded data of the position information and encoded data of the attribute information. In the determination, the three-dimensional data decoding device determines the encoding method used in the encoding of the position information based on the information indicating the encoding method used in the encoding of the position information in the control information included in the encoded stream and included in the first encoding method and the second encoding method, and determines the encoding method used in the encoding of the attribute information based on the information indicating the encoding method used in the encoding of the attribute information in the control information included in the encoded stream and included in the first encoding method and the second encoding method. In the decoding, the three-dimensional data decoding device decodes the encoded data of the position information using the encoding method determined to be used in the encoding of the position information, and decodes the encoded data of the attribute information using the encoding method determined to be used in the encoding of the attribute information.

[0298] Thus, different encoding methods can be used for the position information and the attribute information, so that the encoding efficiency can be improved.

[0299] For example, the encoded stream is stored in one or more units (e.g., NAL units), and the three-dimensional data decoding device further obtains the encoded stream from the one or more units.

[0300] For example, as in Embodiment 1 Figures 15 to 18As described above, the unit includes information which has a format common to the first encoding method and the second encoding method, the information being information indicating the category of data included in the unit and having definitions independent of each other in the first encoding method and the second encoding method (e.g., pcc_nal_unit_type).

[0301] For example, as described in Embodiment 2 Figures 23 to 28 As described above, the unit includes information which has a format independent of each other in the first encoding method and the second encoding method, the information being information indicating the category of data included in the unit and having definitions independent of each other in the first encoding method and the second encoding method (e.g., codec1_nal_unit_type or codec2_nal_unit_type).

[0302] For example, as described in Embodiment 3 Figures 32 to 34 As described above, the unit includes information which has a format common to the first encoding method and the second encoding method, the information being information indicating the category of data included in the unit and having definitions common to each other in the first encoding method and the second encoding method (e.g., pcc_nal_unit_type).

[0303] For example, a three-dimensional data decoding device includes a processor and a memory, and the processor uses the memory to perform the above processing.

[0304] (Embodiment 5)

[0305] In this embodiment, a method for saving NAL units to an ISOBMFF file described in Embodiment 1 will be described.

[0306] ISOBMFF (ISO based media file format) is a file format standard specified by ISO / IEC 14496-12. ISOBMFF specifies a format capable of multiplexing and saving various media such as video, audio, and text, and is a standard independent of the media.

[0307] The basic structure (file) of ISOBMFF will be described. The basic unit in ISOBMFF is a box. A box consists of type, length, and data, and a collection of boxes of various types forms a file.

[0308] Figure 43This is a diagram showing the basic structure (file) of ISOBMFF. The ISOBMFF file mainly includes ftyp which represents the file version (brand) in 4CC (4-character code), moov which stores metadata such as control information, and boxes such as mdat which stores data.

[0309] The storage method for each medium in the ISOBMFF file is specified separately. For example, the storage methods for AVC video and HEVC video are specified by ISO / IEC 14496-15. Here, in order to accumulate or transmit PCC encoded data, it is possible to consider expanding the functions of ISOBMFF for use, but there is no regulation on storing PCC encoded data in the ISOBMFF file. Therefore, in this embodiment, the method for storing PCC encoded data in the ISOBMFF file will be described.

[0310] Figure 44 This is a diagram showing the protocol stack in the case of storing the NAL unit common to the PCC codec in the ISOBMFF file. Here, the NAL unit common to the PCC codec is stored in the ISOBMFF file. The NAL unit is common to the PCC codec, but since multiple PCC codecs are stored in the NAL unit, it is desirable to specify the storage methods (Carriage of Codec1, Carriage of Codec2) corresponding to each codec.

[0311] Next, the method for storing the common PCC NAL unit supporting multiple PCC codecs in the ISOBMFF file will be described. Figure 45 This is a diagram showing an example of storing the ISOBMFF file of the method (Carriage of Codec1) for storing the common PCC NAL unit in Codec 1. Figure 46 This is a diagram showing an example of storing the ISOBMFF file of the method (Carriage of Codec2) for storing the common PCC NAL unit in Codec 2.

[0312] Here, ftyp is important information for identifying the file format. As ftyp, an identifier different for each codec is defined. In the case of storing PCC encoded data encoded by the first encoding method (encoding mode) in the file, ftyp = pcc1 is set. In the case of storing PCC encoded data encoded by the second encoding method in the file, ftyp = pcc2 is set.

[0313] Here, pcc1 represents codec 1 using PCC (the first encoding method). pcc2 represents codec 2 using PCC (the second encoding method). That is, pcc1 and pcc2 indicate that the data is PCC (code data of three-dimensional data (point cloud data)), and also represent PCC codecs (the first encoding method and the second encoding method).

[0314] Hereinafter, a method for saving NAL units to an ISOBMFF file will be described. The multiplexing unit analyzes the NAL unit header, and when pcc_codec_type = Codec1, pcc1 is recorded in the ftyp of ISOBMFF.

[0315] In addition, the multiplexing unit analyzes the NAL unit header, and when pcc_codec_type = Codec2, pcc2 is recorded in the ftyp of ISOBMFF.

[0316] In addition, when pcc_nal_unit_type is metadata, the multiplexing unit saves the NAL unit to moov or mdat, for example, by a prescribed method. When pcc_nal_unit_type is data, the multiplexing unit saves the NAL unit to moov or mdat, for example, by a prescribed method.

[0317] For example, the multiplexing unit can save the NAL unit size to the NAL unit in the same way as HEVC.

[0318] By parsing the ftyp included in the file in the demultiplexing unit (system layer) using this saving method, it is possible to determine whether the PCC encoded data is encoded by the first encoding method or the second encoding method. Furthermore, as described above, by determining whether the PCC encoded data is encoded by the first encoding method or the second encoding method, it is possible to extract the encoded data encoded by one of the encoding methods from the data in which the encoded data encoded by both encoding methods is mixed. Thus, when transmitting the encoded data, the amount of data to be transmitted can be suppressed. In addition, by this saving method, it is possible to use a common data format without setting different data (file) formats for the first encoding method and the second encoding method.

[0319] In addition, when the identification information of the codec is represented in the metadata of the system layer such as ftyp in ISOBMFF, the multiplexing unit can also save the NAL unit after deleting pcc_nal_unit_type to the ISOBMFF file.

[0320] Next, the configurations and operations of the multiplexing unit included in the three-dimensional data encoding system (three-dimensional data encoding device) according to the present embodiment and the demultiplexing unit included in the three-dimensional data decoding system (three-dimensional data decoding device) according to the present embodiment will be described.

[0321] Figure 47 This is a diagram showing the configuration of the first multiplexing unit 4710. The first multiplexing unit 4710 includes a file conversion unit 4711 that generates multiplexed data (file) by saving the encoded data and control information (NAL unit) generated by the first encoding unit 4630 to an ISOBMFF file. The first multiplexing unit 4710 is included in, for example, Figure 1 the multiplexing unit 4614 shown.

[0322] Figure 48 This is a diagram showing the configuration of the first demultiplexing unit 4720. The first demultiplexing unit 4720 includes a file inverse conversion unit 4721 that obtains encoded data and control information (NAL unit) from the multiplexed data (file) and outputs the obtained encoded data and control information to the first decoding unit 4640. The first demultiplexing unit 4720 is included in, for example, Figure 1 the demultiplexing unit 4623 shown.

[0323] Figure 49 This is a diagram showing the configuration of the second multiplexing unit 4730. The second multiplexing unit 4730 includes a file conversion unit 4731 that generates multiplexed data (file) by saving the encoded data and control information (NAL unit) generated by the second encoding unit 4650 to an ISOBMFF file. The second multiplexing unit 4730 is included in, for example, Figure 1 the multiplexing unit 4614 shown.

[0324] Figure 50 This is a diagram showing the configuration of the second demultiplexing unit 4740. The second demultiplexing unit 4740 includes a file inverse conversion unit 4741 that obtains encoded data and control information (NAL unit) from the multiplexed data (file) and outputs the obtained encoded data and control information to the second decoding unit 4660. The second demultiplexing unit 4740 is included in, for example, Figure 1 the demultiplexing unit 4623 shown.

[0325] Figure 51 This is a flowchart of the multiplexing process performed by the first multiplexing unit 4710. First, the first multiplexing unit 4710 determines whether the codec used is the first encoding method or the second encoding method by analyzing the pcc_codec_type included in the NAL unit header (S4701).

[0326] When pcc_codec_type indicates the second encoding method (the second encoding method in S4702), the first multiplexing unit 4710 does not process this NAL unit (S4703).

[0327] On the other hand, when pcc_codec_type indicates the second encoding method (the first encoding method in S4702), the first multiplexing unit 4710 records pcc1 in ftyp (S4704). That is, the first multiplexing unit 4710 records the information indicating that data encoded by the first encoding method is stored in the file in ftyp.

[0328] Next, the first multiplexing unit 4710 analyzes pcc_nal_unit_type included in the NAL unit header, and saves the data to a box (such as moov or mdat) using a specified method corresponding to the data type indicated by pcc_nal_unit_type (S4705). And the first multiplexing unit 4710 creates an ISOBMFF file including the above ftyp and the above box (S4706).

[0329] Figure 52 It is a flowchart of the multiplexing process performed by the second multiplexing unit 4730. First, the second multiplexing unit 4730 determines whether the codec used is the first encoding method or the second encoding method by analyzing pcc_codec_type included in the NAL unit header (S4711).

[0330] When pcc_unit_type indicates the second encoding method (the second encoding method in S4712), the second multiplexing unit 4730 records pcc2 in ftyp (S4713). That is, the second multiplexing unit 4730 records the information indicating that data encoded by the second encoding method is stored in the file in ftyp.

[0331] Next, the second multiplexing unit 4730 analyzes pcc_nal_unit_type included in the NAL unit header, and saves the data to a box (such as moov or mdat) using a specified method corresponding to the data type indicated by pcc_nal_unit_type (S4714). And the second multiplexing unit 4730 creates an ISOBMFF file including the above ftyp and the above box (S4715).

[0332] On the other hand, when pcc_unit_type indicates the first encoding method (the first encoding method in S4712), the second multiplexing unit 4730 does not process this NAL unit (S4716).

[0333] In addition, the above processing shows an example in which PCC data is encoded using either the first encoding method or the second encoding method. The first multiplexing unit 4710 and the second multiplexing unit 4730 save the NAL unit desired for the file by identifying the codec type of the NAL unit. In addition, when identification information of the PCC codec is included outside the NAL unit header, the first multiplexing unit 4710 and the second multiplexing unit 4730 may also use the identification information of the PCC codec included outside the NAL unit header to identify the codec type (the first encoding method or the second encoding method) in steps S4701 and S4711.

[0334] In addition, the first multiplexing unit 4710 and the second multiplexing unit 4730 may also save the data to the file in steps S4706 and S4714 after deleting pcc_nal_unit_type from the NAL unit header and then saving it to the file.

[0335] Figure 53 It is a flowchart showing the processing performed by the first demultiplexing unit 4720 and the first decoding unit 4640. First, the first demultiplexing unit 4720 analyzes ftyp included in the ISOBMFF file (S4721). When the codec represented by ftyp is the second encoding method (pcc2) (it is the second encoding method in S4722), the first demultiplexing unit 4720 determines that the data included in the payload of the NAL unit is data encoded using the second encoding method (S4723). In addition, the first demultiplexing unit 4720 passes the determination result to the first decoding unit 4640. The first decoding unit 4640 does not process this NAL unit (S4724).

[0336] On the other hand, when the codec represented by ftyp is the first encoding method (pcc1) (it is the first encoding method in S4722), the first demultiplexing unit 4720 determines that the data included in the payload of the NAL unit is data encoded using the first encoding method (S4725). In addition, the first demultiplexing unit 4720 passes the determination result to the first decoding unit 4640.

[0337] The first decoding unit 4640 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first encoding method. And the first decoding unit 4640 decodes the PCC data using the decoding process of the first encoding method (S4727).

[0338] Figure 54This is a flowchart showing the processes performed by the second demultiplexing unit 4740 and the second decoding unit 4660. First, the second demultiplexing unit 4740 parses the ftyp included in the ISOBMFF file (S4731). When the codec represented by the ftyp is the second encoding method (pcc2) (Yes in S4732 for the second encoding method), the second demultiplexing unit 4740 determines that the data included in the payload of the NAL unit is data encoded by the second encoding method (S4733). In addition, the second demultiplexing unit 4740 passes the determination result to the second decoding unit 4660.

[0339] The second decoding unit 4660 identifies the data by setting the pcc_nal_unit_type included in the NAL unit header as the identifier of the NAL unit for the second encoding method (S4734). And the second decoding unit 4660 decodes the PCC data using the decoding process of the second encoding method (S4735).

[0340] On the other hand, when the codec represented by the ftyp is the first encoding method (pcc1) (Yes in S4732 for the first encoding method), the second demultiplexing unit 4740 determines that the data included in the payload of the NAL unit is data encoded by the first encoding method (S4736). In addition, the second demultiplexing unit 4740 passes the determination result to the second decoding unit 4660. The second decoding unit 4660 does not process this NAL unit (S4737).

[0341] In this way, for example, by identifying the codec type of the NAL unit in the first demultiplexing unit 4720 or the second demultiplexing unit 4740, the codec type can be identified at an earlier stage. Furthermore, the desired NAL unit can be input to the first decoding unit 4640 or the second decoding unit 4660, while removing the unnecessary NAL units. In this case, in the first decoding unit 4640 or the second decoding unit 4660, it may not be necessary to perform the process of parsing the identification information of the codec. Additionally, the process of parsing the identification information of the codec by referring to the NAL unit type again in the first decoding unit 4640 or the second decoding unit 4660 can also be implemented.

[0342] In addition, when the pcc_nal_unit_type is deleted from the NAL unit header in the first multiplexing unit 4710 or the second multiplexing unit 4730, the first demultiplexing unit 4720 or the second demultiplexing unit 4740 may also output to the first decoding unit 4640 or the second decoding unit 4660 after assigning the pcc_nal_unit_type to the NAL unit.

[0343] (Embodiment 6)

[0344] In the present embodiment, the multiplexing unit and the demultiplexing unit corresponding to the encoding unit 4670 and the decoding unit 4680 corresponding to a plurality of codecs described in Embodiment 4 will be described. Figure 55 It is a diagram showing the configuration of the encoding unit 4670 and the third multiplexing unit 4750 according to the present embodiment.

[0345] The encoding unit 4670 encodes the point cloud data by using one or both of the first encoding method and the second encoding method. The encoding unit 4670 may switch the encoding method (the first encoding method and the second encoding method) in units of point cloud data or frames. In addition, the encoding unit 4670 may switch the encoding method in units that can be encoded.

[0346] The encoding unit 4670 generates encoded data (encoded stream) including the identification information of the PCC codec described in Embodiments 1 to 4.

[0347] The third multiplexing unit 4750 includes a file conversion unit 4751. The file conversion unit 4751 converts the NAL unit output from the encoding unit 4670 into a file of PCC data. The file conversion unit 4751 analyzes the codec identification information included in the NAL unit header, and determines whether the PCC encoded data is data encoded by the first encoding method, data encoded by the second encoding method, or data encoded by both methods. The file conversion unit 4751 records the version name capable of identifying the codec in the ftyp. For example, in the case of indicating encoding by both methods, pcc3 is recorded in the ftyp.

[0348] In addition, when the encoding unit 4670 records the identification information of the PCC codec outside the NAL unit, the file conversion unit 4751 may also use this identification information to determine the PCC codec (encoding method).

[0349] Figure 56 It is a diagram showing the configuration of the third demultiplexing unit 4760 and the decoding unit 4680 according to the present embodiment.

[0350] The third demultiplexing unit 4760 includes a file inverse conversion unit 4761. The file inverse conversion unit 4761 analyzes the ftyp included in the file, and determines whether the PCC encoded data is data encoded by the first encoding method, data encoded by the second encoding method, or data encoded by both methods.

[0351] When the PCC encoded data is encoded by one of the encoding methods, the data is input to the corresponding decoding unit among the first decoding unit 4640 and the second decoding unit 4660, and the data is not input to the other decoding unit. When the PCC encoded data is encoded by both encoding methods, the data is input to the decoding unit 4680 corresponding to both methods.

[0352] The decoding unit 4680 decodes the PCC encoded data in a manner using one or both of the first encoding method and the second encoding method.

[0353] Figure 57 It is a flowchart showing the processing performed by the third multiplexing unit 4750 according to the present embodiment.

[0354] First, the third multiplexing unit 4750 determines whether the codec used is the first encoding method, the second encoding method, or both the first encoding method and the second encoding method by analyzing the pcc_codec_type included in the NAL unit header (S4741).

[0355] When the second encoding method is used (Yes in S4742 and it is the second encoding method in S4743), the third multiplexing unit 4750 records pcc2 in the ftyp (S4744). That is, the third multiplexing unit 4750 records information indicating that data encoded using the second encoding method is stored in the file in the ftyp.

[0356] Next, the third multiplexing unit 4750 analyzes the pcc_nal_unit_type included in the NAL unit header and saves the data to a box (such as moov or mdat) using a prescribed method corresponding to the data type indicated by the pcc_unit_type (S4745). And the third multiplexing unit 4750 creates an ISOBMFF file including the above ftyp and the above box (S4746).

[0357] On the other hand, when the first encoding method is used (Yes in S4742 and it is the first encoding method in S4743), the third multiplexing unit 4750 records pcc1 in the ftyp (S4747). That is, the third multiplexing unit 4750 records information indicating that data encoded using the first encoding method is stored in the file in the ftyp.

[0358] Next, the third multiplexing unit 4750 analyzes the pcc_nal_unit_type included in the NAL unit header and saves the data to a box (such as moov or mdat) using a prescribed method corresponding to the data type indicated by the pcc_unit_type (S4748). And the third multiplexing unit 4750 creates an ISOBMFF file including the above ftyp and the above box (S4746).

[0359] On the other hand, in the case where both the first encoding method and the second encoding method are used (No in S4742), the third multiplexing unit 4750 records pcc3 in ftyp (S4749). That is, the third multiplexing unit 4750 records information indicating that data encoded using both encoding methods is stored in the file in ftyp.

[0360] Next, the third multiplexing unit 4750 analyzes pcc_nal_unit_type included in the NAL unit header, and saves the data to a box (such as moov or mdat) using a prescribed method corresponding to the data type indicated by pcc_unit_type (S4750). And the third multiplexing unit 4750 creates an ISOBMFF file including the above ftyp and the above box (S4746).

[0361] Figure 58 It is a flowchart showing the processing performed by the third demultiplexing unit 4760 and the decoding unit 4680. First, the third demultiplexing unit 4760 analyzes ftyp included in the ISOBMFF file (S4761). When the codec indicated by ftyp is the second encoding method (pcc2) (Yes in S4762 and it is the second encoding method in S4763), the third demultiplexing unit 4760 determines that the data included in the payload of the NAL unit is data encoded using the second encoding method (S4764). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0362] The decoding unit 4680 identifies the data by setting pcc_nal_unit_type included in the NAL unit header as an identifier of a NAL unit for the second encoding method. And the decoding unit 4680 decodes the PCC data using the decoding process of the second encoding method (S4766).

[0363] On the other hand, when the codec indicated by ftyp is the first encoding method (pcc1) (Yes in S4762 and it is the first encoding method in S4763), the third demultiplexing unit 4760 determines that the data included in the payload of the NAL unit is data encoded using the first encoding method (S4767). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0364] The decoding unit 4680 identifies the data by setting pcc_nal_unit_type included in the NAL unit header as an identifier of a NAL unit for the first encoding method. And the decoding unit 4680 decodes the PCC data using the decoding process of the first encoding method (S4769).

[0365] On the other hand, when the coding method (pcc3) used by both parties is indicated by ftyp (No in S4762), the third demultiplexing unit 4760 determines that the data included in the payload of the NAL unit is data encoded by both the first coding method and the second coding method (S4770). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0366] The decoding unit 4680 identifies the data by setting the pcc_nal_unit_type included in the NAL unit header as the identifier of the NAL unit for the codec described in pcc_codec_type (S4771). And the decoding unit 4680 decodes the PCC data using the decoding processes of both coding methods (S4772). That is, the decoding unit 4680 decodes the data encoded by the first coding method using the decoding process of the first coding method, and decodes the data encoded by the second coding method using the decoding process of the second coding method.

[0367] Hereinafter, a modification example of the present embodiment will be described. As the types of versions indicated by ftyp, the following types may also be represented by identification information. In addition, a combination of multiple types shown below may also be represented by identification information.

[0368] The identification information indicates whether the object of the original data before PCC coding is a point cloud with a restricted area or a large-scale point cloud without a restricted area such as map information.

[0369] The identification information may also indicate whether the original data before PCC coding is a static object or a dynamic object.

[0370] As described above, the identification information may also indicate whether the PCC encoded data is data encoded by the first coding method or data encoded by the second coding method.

[0371] The identification information may also indicate the algorithm used in PCC coding. Here, the algorithm is, for example, a coding method that can be used in the first coding method or the second coding method.

[0372] The identification information may also indicate the difference in the method of saving the PCC encoded data to an ISOBMFF file. For example, the identification information may indicate whether the saving method used is a saving method for accumulation or a saving method for real-time transmission such as dynamic streaming.

[0373] In addition, in Embodiment 5, the method of storing NAL units described in Embodiment 1 was explained. In Embodiment 6, the method of storing NAL units described in Embodiment 4 was explained. However, the same storage method can also be applied to the method of storing NAL units described in Embodiments 2 and 3, and thereby the identification information of the PCC codec can be stored in an ISOBMFF file.

[0374] In addition, in Embodiments 5 and 6, an example of using ISOBMFF as a file format was explained. However, other formats can also be used. For example, the same method as in this embodiment can also be used when storing PCC encoded data in MPEG-2TS Systems, MPEG-DASH, MMT, or RMP.

[0375] In addition, in the above, an example of storing metadata such as identification information in ftyp was shown. However, these metadata can also be stored other than in ftyp. For example, these metadata can be stored in moov.

[0376] As described above, the three-dimensional data storage device (or three-dimensional data multiplexing device, or three-dimensional data encoding device) performs Figure 59 the processing shown.

[0377] First, the three-dimensional data storage device (for example, including the first multiplexing unit 4710, the second multiplexing unit 4730, or the third multiplexing unit 4750) obtains one or more units (for example, NAL units) that store an encoded stream obtained by encoding point cloud data (S4781). Next, the three-dimensional data storage device stores the one or more units in a file (for example, an ISOBMFF file) (S4782). In addition, in the above storage (S4782), the three-dimensional data storage device stores information (for example, pcc1, pcc2, or pcc3) indicating that the data stored in the file is data obtained by encoding point cloud data in the control information (for example, ftyp) of the above file.

[0378] Thereby, in a device that processes the file generated by this three-dimensional data storage device, it is possible to refer to the control information of the file and determine earlier whether the data stored in the file is encoded data of point cloud data. Therefore, it is possible to reduce the processing amount of the device or speed up the processing.

[0379] For example, the above information also represents the encoding methods used in the encoding of the above point cloud data in the first encoding method and the second encoding method. In addition, the data stored in the file, which is the data obtained by encoding the point cloud data and the encoding methods used in the encoding of the point cloud data in the first encoding method and the second encoding method, can be represented by a single piece of information or by different pieces of information.

[0380] Thus, in a device that processes the file generated by this three-dimensional data storage device, it is possible to refer to the control information of the file and determine earlier the codec used for the data stored in the file. Therefore, it is possible to reduce the processing amount of the device or speed up the processing.

[0381] For example, the above first encoding method is a method (GPCC) of encoding the position information representing the position of the point cloud data using an N-ary tree (where N is an integer of 2 or more) and encoding the attribute information using the above position information, and the above second encoding method is a method (VPCC) of generating a two-dimensional image from the point cloud data and encoding the above two-dimensional image using an image encoding method.

[0382] For example, the above file conforms to ISOBMFF (ISO based media file format).

[0383] For example, the three-dimensional data storage device includes a processor and a memory, and the processor uses the memory to perform the above processing.

[0384] In addition, as described above, the three-dimensional data acquisition device (or three-dimensional data demultiplexing device, or three-dimensional data decoding device) performs Figure 60 the processing shown.

[0385] The three-dimensional data acquisition device (for example, including a first demultiplexing unit 4720, a second demultiplexing unit 4740, or a third demultiplexing unit 4760) acquires a file (for example, an ISOBMFF file) that stores one or more units (for example, NAL units), and the one or more units store an encoded stream obtained by encoding the point cloud data (S4791). Next, the three-dimensional data acquisition device acquires one or more units from the file (S4792). In addition, the control information of the file (for example, ftyp) includes information (for example, pcc1, pcc2, or pcc3) indicating that the data stored in the file is data obtained by encoding the point cloud data.

[0386] For example, the three-dimensional data acquisition device refers to the above information to determine whether the data stored in the file is data obtained by encoding point cloud data. Further, when the three-dimensional data acquisition device determines that the data stored in the file is data obtained by encoding point cloud data, it generates point cloud data by decoding the data obtained by encoding point cloud data included in one or more units. Alternatively, when the three-dimensional data acquisition device determines that the data stored in the file is data obtained by encoding point cloud data, it outputs (notifies) information indicating that the data included in one or more units is data obtained by encoding point cloud data to a subsequent processing unit (for example, the first decoding unit 4640, the second decoding unit 4660, or the decoding unit 4680).

[0387] Thereby, the three-dimensional data acquisition device can refer to the control information of the file to determine earlier whether the data stored in the file is encoded data of point cloud data. Therefore, it is possible to reduce the processing amount or speed up the processing of the three-dimensional data acquisition device or the subsequent device.

[0388] For example, the above information also indicates the encoding method used in the above encoding among the first encoding method and the second encoding method. In addition, the data stored in the file is data obtained by encoding point cloud data, and the encoding method used in the encoding of point cloud data among the first encoding method and the second encoding method may be represented by a single piece of information or by different pieces of information.

[0389] Thereby, the three-dimensional data acquisition device can refer to the control information of the file to determine earlier the codec used for the data stored in the file. Therefore, it is possible to reduce the processing amount or speed up the processing of the three-dimensional data acquisition device or the subsequent device.

[0390] For example, the three-dimensional data acquisition device obtains data encoded by a certain encoding method from the encoded point cloud data including data encoded by the first encoding method and data encoded by the second encoding method based on the above information.

[0391] For example, the above first encoding method is a method (GPCC) of encoding position information representing the position of point cloud data with an N-ary tree (N is an integer of 2 or more) and encoding attribute information using the above position information, and the above second encoding method is a method (VPCC) of generating a two-dimensional image from point cloud data and encoding the two-dimensional image using an image encoding method.

[0392] For example, the above file is based on ISOBMFF (ISO based media file format).

[0393] For example, the three-dimensional data acquisition device includes a processor and a memory, and the processor uses the memory to perform the above processing.

[0394] (Embodiment 7)

[0395] In the present embodiment, the types of encoded data (Geometry, Attribute, Metadata) generated by the above-described first encoding unit 4630 or second encoding unit 4650, the generation method of the additional information (metadata), and the multiplexing process in the multiplexing unit will be described. In addition, the additional information (metadata) may also be referred to as a parameter set or control information in some cases.

[0396] In the present embodiment, taking the dynamic object (three-dimensional point cloud data that changes over time) described in Figure 4 as an example, the same method can also be used in the case of a static object (three-dimensional point cloud data at any moment).

[0397] Figure 61 is a diagram showing the configuration of the encoding unit 4801 and the multiplexing unit 4802 included in the three-dimensional data encoding device according to the present embodiment. The encoding unit 4801 corresponds to, for example, the above-described first encoding unit 4630 or second encoding unit 4650. The multiplexing unit 4802 corresponds to the above-described multiplexing units 4634 or 46456.

[0398] The encoding unit 4801 encodes the point cloud data of multiple PCC (Point Cloud Compression) frames to generate encoded data (Multiple Compressed Data) of multiple position information, attribute information, and additional information.

[0399] The multiplexing unit 4802 transforms the data into a data structure considering data access in the decoding device by NAL unitizing the data of multiple data types (position information, attribute information, and additional information).

[0400] Figure 62 is a diagram showing a configuration example of the encoded data generated by the encoding unit 4801. The arrows in the figure represent the dependency relationships related to the decoding of the encoded data, and the root of the arrow depends on the data at the tip of the arrow. That is, the decoding device decodes the data at the tip of the arrow and uses the decoded data to decode the data at the root of the arrow. In other words, dependency means referring to (using) the data of the dependency target in the processing (encoding or decoding, etc.) of the data of the dependency source.

[0401] First, the generation process of the encoded data of the position information will be described. The encoding unit 4801 generates the encoded position data (Compressed Geometry Data) of each frame by encoding the position information of each frame. In addition, the encoded position data is represented by G(i). Here, i represents the frame number, the time of the frame, or the like.

[0402] In addition, the encoding unit 4801 generates a set of position parameters (GPS(i)) corresponding to each frame. The set of position parameters includes parameters that can be used in the decoding of the encoded position data. In addition, the encoded position data of each frame depends on the corresponding set of position parameters.

[0403] In addition, the encoded position data composed of multiple frames is defined as a geometry sequence (Geometry Sequence). The encoding unit 4801 generates a set of geometry sequence parameters (Geometry Sequence PS: also denoted as Position SPS) that stores the parameters that will be commonly used in the decoding process for multiple frames within the geometry sequence. The geometry sequence depends on the Position SPS.

[0404] Next, the generation process of the encoded data of the attribute information will be described. The encoding unit 4801 generates the encoded attribute data (Compressed Attribute Data) of each frame by encoding the attribute information of each frame. In addition, the encoded attribute data is represented by A(i). In addition, in Figure 62 an example where there are Attribute X and Attribute Y is shown. The encoded attribute data of Attribute X is represented by AX(i), and the encoded attribute data of Attribute Y is represented by AY(i).

[0405] In addition, the encoding unit 4801 generates a set of attribute parameters (APS(i)) corresponding to each frame. In addition, the set of attribute parameters of Attribute X is represented by AXPS(i), and the set of attribute parameters of Attribute Y is represented by AYPS(i). The set of attribute parameters includes parameters that can be used in the decoding of the encoded attribute information. The encoded attribute data depends on the corresponding set of attribute parameters.

[0406] In addition, the encoded attribute data composed of multiple frames is defined as an attribute sequence (Attribute Sequence). The encoding unit 4801 generates a set of attribute sequence parameters (Attribute Sequence PS: also denoted as Attribute SPS) that stores the parameters that will be commonly used in the decoding process for multiple frames within the attribute sequence. The attribute sequence depends on the Attribute SPS.

[0407] In addition, in the first encoding method, the encoded attribute data depends on the encoded position data.

[0408] In addition, in Figure 62In [example], an example is shown in which there are two types of attribute information (attribute X and attribute Y). In the case of having two types of attribute information, for example, respective data and metadata are generated by two encoding units. In addition, for example, an attribute sequence is defined for each type of attribute information, and an attribute SPS is generated for each type of attribute information.

[0409] In addition, in Figure 62 an example is shown in which there is one type of position information and two types of attribute information, but it is not limited to this. The attribute information may also be one type or three or more types. In this case, the encoded data can also be generated by the same method. In addition, in the case of point cloud data without attribute information, there may be no attribute information. In this case, the encoding unit 4801 may not generate a parameter set associated with the attribute information.

[0410] Next, the generation process of the additional information (metadata) will be described. The encoding unit 4801 generates a parameter set for the entire PCC stream, that is, the PCC stream PS (PCC Stream PS: also denoted as stream PS). The encoding unit 4801 stores in the stream PS parameters that can be commonly used in the decoding process of one or more position sequences and one or more attribute sequences. For example, in the stream PS, it includes identification information indicating the codec of the point cloud data and information indicating the algorithm used in the encoding, etc. The position sequence and the attribute sequence depend on the stream PS.

[0411] Next, the access unit and the GOF will be described. In the present embodiment, a consideration method of a new access unit (AccessUnit: AU) and a GOF (Group of Frame) is introduced.

[0412] The access unit is the basic unit used to access data during decoding and is composed of one or more data and one or more metadata. For example, the access unit is composed of position information at the same time and one or more attribute information. The GOF is a random access unit and is composed of one or more access units.

[0413] The encoding unit 4801 generates an access unit header (AU Header) as identification information indicating the start of the access unit. The encoding unit 4801 stores in the access unit header parameters related to the access unit. For example, the access unit header includes: the composition or information of the encoded data included in the access unit. In addition, the access unit header includes parameters commonly used for the data included in the access unit, such as parameters related to the decoding of the encoded data, etc.

[0414] In addition, the encoding unit 4801 may also generate an access unit delimiter that does not include parameters related to the access unit instead of the access unit header. This access unit delimiter is used as identification information indicating the start of the access unit. The decoding device identifies the start of the access unit by detecting the access unit header or the access unit delimiter.

[0415] Next, the generation of the identification information at the start of the GOF will be described. The encoding unit 4801 generates a GOF header (GOFHeader) as the identification information indicating the start of the GOF. The encoding unit 4801 stores parameters related to the GOF in the GOF header. For example, the GOF header includes: the composition or information of the encoded data included in the GOF. In addition, the GOF header includes parameters commonly used for the data included in the GOF, such as parameters related to the decoding of the encoded data, etc.

[0416] In addition, the encoding unit 4801 may also generate a GOF delimiter that does not include parameters related to the GOF instead of the GOF header. This GOF delimiter is used as identification information indicating the start of the GOF. The decoding device identifies the start of the GOF by detecting the GOF header or the GOF delimiter.

[0417] In the PCC encoded data, for example, it is defined that the access unit is in units of PCC frames. The decoding device accesses the PCC frames based on the identification information at the start of the access unit.

[0418] In addition, for example, the GOF is defined as one random access unit. The decoding device accesses the random access unit based on the identification information at the start of the GOF. For example, if the PCC frames are independent of each other and can be decoded individually, the PCC frames can also be defined as random access units.

[0419] In addition, two or more PCC frames may be assigned to one access unit, and multiple random access units may be assigned to one GOF.

[0420] In addition, the encoding unit 4801 may also define and generate parameter sets or metadata other than those described above. For example, the encoding unit 4801 may generate SEI (Supplemental Enhancement Information) that stores parameters (optional parameters) that may not necessarily be used during decoding.

[0421] Next, the composition of the encoded data and the method of storing the encoded data in the NAL unit will be described.

[0422] For example, the data format is specified for each type of encoded data. Figure 63 It is a diagram showing examples of the encoded data and the NAL unit.

[0423] For example, asFigure 63 As shown, the encoded data includes a header and a payload. Additionally, the encoded data may also include encoded data, a header, or length information indicating the length (amount of data) of the payload. Furthermore, the encoded data may not include a header.

[0424] The header contains, for example, identification information used to determine the data. This identification information represents, for example, the data type or frame number.

[0425] The header contains, for example, identification information indicating a reference relationship. This identification information is saved in the header when there is a dependency between data and is used to reference the reference target from the reference source. For example, in the header of the reference target, the identification information used to determine the data is included. In the header of the reference source, the identification information indicating the reference target is included.

[0426] Additionally, when the reference target or reference source can be identified or derived from other information, the identification information used to determine the data or the identification information indicating the reference relationship may be omitted.

[0427] The multiplexing unit 4802 stores the encoded data in the payload of the NAL unit. In the NAL unit header, pcc_nal_unit_type, which is the identification information of the encoded data, is included. Figure 64 It is a diagram showing semantic examples of pcc_nal_unit_type.

[0428] As Figure 64 shown, when pcc_codec_type is Codec 1 (the first encoding method), the values 0 to 10 of pcc_nal_unit_type are assigned to the encoded position data (Geometry), encoded attribute X data (AttributeX), encoded attribute Y data (AttributeY), position PS (Geom.PS), attribute XPS (AttrX.PS), attribute YPS (AttrX.PS), position SPS (Geometry Sequence PS), attribute XSPS (AttributeX Sequence PS), attribute YSPS (AttributeY Sequence PS), AU header (AU Header), and GOF header (GOF Header) in Codec 1. Additionally, values 11 and later are assigned as spares for Codec 1.

[0429] When pcc_codec_type is Codec2 (the second encoding method), values 0 to 2 of pcc_nal_unit_type are assigned to Data A, MetaData A, and MetaData B of the codec. In addition, values 3 and later are assigned as reserves for Codec2.

[0430] Next, the data transmission order will be described. Hereinafter, the restrictions on the NAL unit transmission order will be described.

[0431] The multiplexing unit 4802 transmits NAL units together in units of GOF or AU. The multiplexing unit 4802 configures the GOF header at the beginning of the GOF and the AU header at the beginning of the AU.

[0432] The multiplexing unit 4802 can also configure the sequence parameter set (SPS) for each AU so that the decoding device can start decoding from the next AU even if data is lost due to packet loss or the like.

[0433] When there is a decoding-dependent relationship in the encoded data, the decoding device decodes the data of the reference source after decoding the data of the reference target. In the decoding device, in order to be able to decode in the received order without rearranging the data, the multiplexing unit 4802 transmits the data of the reference target first.

[0434] Figure 65 It is a diagram showing an example of the NAL unit transmission order. Figure 65 It shows three examples: position information priority, parameter priority, and data merging.

[0435] The transmission order of position information priority is an example of transmitting each piece of information related to position information and each piece of information related to attribute information together. In this transmission order, the transmission of information related to position information is completed earlier than the transmission of information related to attribute information.

[0436] For example, by using this transmission order, a decoding device that does not decode attribute information may be able to set the time without processing by ignoring the decoding of attribute information. In addition, for example, in the case of a decoding device that wants to decode position information earlier, it may be able to decode position information earlier by obtaining the encoded data of position information earlier.

[0437] In addition, in Figure 65 the attribute XSPS and the attribute YSPS are merged and denoted as the attribute SPS, but the attribute XSPS and the attribute YSPS can also be configured separately.

[0438] In the transmission order of parameter set priority, the parameter set is transmitted first, and then the data is transmitted.

[0439] As described above, as long as the constraints of the NAL unit transmission order are followed, the multiplexing unit 4802 can transmit the NAL units in any order. For example, order identification information can be defined, and the multiplexing unit 4802 has the function of transmitting NAL units in multiple styles of order. For example, the order identification information of the NAL units is saved in the stream PS.

[0440] The 3D data decoding device can also perform decoding based on the order identification information. It is also possible to indicate the desired transmission order from the 3D data decoding device to the 3D data encoding device, and the 3D data encoding device (multiplexing unit 4802) controls the transmission order according to the indicated transmission order.

[0441] In addition, as long as it is within the range that follows the constraints of the transmission order such as the transmission order of data merging, the multiplexing unit 4802 can also generate encoded data that merges multiple functions. For example, as Figure 65 shown, it is also possible to merge the GOF header and the AU header, and it is also possible to merge AXPS and AYPS. In this case, in the pcc_nal_unit_type, an identifier indicating that it is data with multiple functions is defined.

[0442] Hereinafter, a modification example of the present embodiment will be described. PS has levels such as frame-level PS, sequence-level PS, and PCC sequence-level PS. If the PCC sequence-level is set as the upper level and the frame-level is set as the lower level, the following method can also be used for the parameter saving method.

[0443] The default PS value is represented by the higher-level PS. In addition, when the value of the lower-level PS is different from the value of the higher-level PS, the value of PS is represented by the lower-level PS. Or, the value of PS is not recorded in the upper level, and the value of PS is recorded in the lower-level PS. Or, information indicating whether the value of PS is represented by the lower-level PS, the higher-level PS, or both is represented by one or both of the lower-level PS and the higher-level PS. Or, the lower-level PS can be merged into the higher-level PS. Or, when the lower-level PS and the higher-level PS overlap, the multiplexing unit 4802 can also omit the transmission of one of them.

[0444] In addition, the encoding unit 4801 or the multiplexing unit 4802 can also divide the data into slices or tiles, etc., and transmit the divided data. The divided data contains information for identifying the divided data, and the parameter set contains parameters used in the decoding of the divided data. In this case, in the pcc_nal_unit_type, an identifier indicating that it is data related to tiles or slices or data for saving parameters is defined.

[0445] Hereinafter, the processing of the sequence identification information will be described. Figure 66 It is a flowchart of the processing performed by the three-dimensional data encoding device (encoding unit 4801 and multiplexing unit 4802) related to the transmission order of NAL units.

[0446] First, the three-dimensional data encoding device determines the transmission order of NAL units (position information first or parameter set first) (S4801). For example, the three-dimensional data encoding device determines the transmission order based on a specification from a user or an external device (e.g., a three-dimensional data decoding device).

[0447] When the determined transmission order is position information first (position information first in S4802), the three-dimensional data encoding device sets the sequence identification information included in the stream PS to position information first (S4803). That is, in this case, the sequence identification information indicates that NAL units are transmitted in the order of position information first. And, the three-dimensional data encoding device transmits NAL units in the order of position information first (S4804).

[0448] On the other hand, when the determined transmission order is parameter set first (parameter set first in S4802), the three-dimensional data encoding device sets the sequence identification information included in the stream PS to parameter set first (S4805). That is, in this case, the sequence identification information indicates that NAL units are transmitted in the order of parameter set first. And, the three-dimensional data encoding device transmits NAL units in the order of parameter set first (S4806).

[0449] Figure 67 It is a flowchart of the processing performed by the three-dimensional data decoding device related to the transmission order of NAL units. First, the three-dimensional data decoding device analyzes the sequence identification information included in the stream PS (S4811).

[0450] When the transmission order indicated by the sequence identification information is position information first (position information first in S4812), the three-dimensional data decoding device sets the transmission order of NAL units to position information first and decodes the NAL units (S4813).

[0451] On the other hand, when the transmission order indicated by the sequence identification information is parameter set first (parameter set first in S4812), the three-dimensional data decoding device sets the transmission order of NAL units to parameter set first and decodes the NAL units (S4814).

[0452] For example, in step S4813, the three-dimensional data decoding device may obtain NAL units related to position information without obtaining all NAL units and decode the position information from the obtained NAL units without decoding the attribute information.

[0453] Next, the processing related to the generation of AUs and GOFs will be described. Figure 68 It is a flowchart showing the processing performed by the three-dimensional data encoding device (multiplexing unit 4802) related to the generation of AUs and GOFs in the multiplexing of NAL units.

[0454] First, the three-dimensional data encoding device determines the type of encoded data (S4821). Specifically, the three-dimensional data encoding device determines whether the encoded data to be processed is data starting with an AU, data starting with a GOF, or other data.

[0455] When the encoded data is data starting with a GOF (GOF start in S4822), the three-dimensional data encoding device generates an NAL unit by arranging the GOF header and the AU header at the beginning of the encoded data belonging to the GOF (S4823).

[0456] When the encoded data is data starting with an AU (AU start in S4822), the three-dimensional data encoding device generates an NAL unit by arranging the AU header at the beginning of the encoded data belonging to the AU (S4824).

[0457] When the encoded data is neither starting with a GOF nor starting with an AU (other than GOF start and AU start in S4822), the three-dimensional data encoding device generates an NAL unit by arranging the encoded data after the AU header of the AU to which the encoded data belongs (S4825).

[0458] Next, the processing related to the access to AUs and GOFs will be described. Figure 69 It is a flowchart showing the processing of the three-dimensional data decoding device related to the access to AUs and GOFs in the demultiplexing of NAL units.

[0459] First, the three-dimensional data decoding device determines the type of encoded data included in the NAL unit by analyzing the nal_unit_type included in the NAL unit (S4831). Specifically, the three-dimensional data decoding device determines whether the encoded data included in the NAL unit is data starting with an AU, data starting with a GOF, or other data.

[0460] When the encoded data included in the NAL unit is data starting with a GOF (GOF start in S4832), the three-dimensional data decoding device determines that the NAL unit is the start position of random access, accesses the NAL unit, and starts the decoding process (S4833).

[0461] On the other hand, when the encoded data included in the NAL unit is data starting with AU (starting with AU in S4832), the three-dimensional data decoding device determines that the NAL unit starts with AU, accesses the data included in the NAL unit, and decodes the AU (S4834).

[0462] On the other hand, when the encoded data included in the NAL unit is not any of the data starting with GOF or AU (other than starting with GOF or AU in S4832), the three-dimensional data decoding device does not process the NAL unit.

[0463] As described above, the three-dimensional data encoding device performs Figure 70 the processing shown. The three-dimensional data encoding device encodes time-series three-dimensional data (for example, point cloud data of a dynamic object). The three-dimensional data includes position information and attribute information at each moment.

[0464] First, the three-dimensional data encoding device encodes the position information (S4841). Next, the three-dimensional data encoding device encodes the attribute information of the processing object with reference to the position information at the same moment as the attribute information of the processing object (S4842). Here, as Figure 62 shown, the position information and the attribute information at the same moment constitute an access unit (AU). That is, the three-dimensional data encoding device encodes the attribute information of the processing object with reference to the position information included in the same access unit as the attribute information of the processing object.

[0465] Thereby, the three-dimensional data encoding device can facilitate the control of the reference in the encoding using the access unit. Therefore, the three-dimensional data encoding device can reduce the processing amount of the encoding process.

[0466] For example, the three-dimensional data encoding device generates a bitstream including the encoded position information (encoded position data), the encoded attribute information (encoded attribute data), and the position information indicating the reference target of the attribute information of the processing object.

[0467] For example, the bitstream includes: a position parameter set (position PS) of control information including the position information at each moment, and an attribute parameter set (attribute PS) of control information including the attribute information at each moment.

[0468] For example, the bitstream includes: a position sequence parameter set (position SPS) of control information common to the position information at multiple moments, and an attribute sequence parameter set (attribute SPS) of control information common to the attribute information at multiple moments.

[0469] For example, the bitstream includes: a stream parameter set (stream PS) of control information common to the position information at multiple moments and the attribute information at multiple moments.

[0470] For example, the bitstream includes an access unit header (AU header) that includes control information common within an access unit.

[0471] For example, the three-dimensional data encoding device encodes a group of pictures (GOP) composed of one or more access units so that it can be decoded independently. That is, the GOP is a random access unit.

[0472] For example, the bitstream includes a GOP header that includes control information common within the GOP.

[0473] For example, the three-dimensional data encoding device includes a processor and a memory, and the processor uses the memory to perform the above processing.

[0474] In addition, as described above, the three-dimensional data decoding device performs Figure 71 the processing shown. The three-dimensional data decoding device decodes time-series three-dimensional data (for example, point cloud data of a dynamic object). The three-dimensional data includes position information and attribute information at each time. The position information and attribute information at the same time constitute an access unit (AU).

[0475] First, the three-dimensional data decoding device decodes the position information from the bitstream (S4851). That is, the three-dimensional data decoding device generates position information by decoding the encoded position information (encoded position data) included in the bitstream.

[0476] Next, the three-dimensional data decoding device decodes the attribute information of the processing object from the bitstream with reference to the position information at the same time as the attribute information of the processing object (S4852). That is, the three-dimensional data decoding device generates attribute information by decoding the encoded attribute information (encoded attribute data) included in the bitstream. At this time, the three-dimensional data decoding device refers to the decoded position information included in the same access unit as the attribute information.

[0477] Thereby, the three-dimensional data decoding device can use the access unit to facilitate the reference in decoding. Therefore, this three-dimensional data decoding method can reduce the processing amount of the decoding process.

[0478] For example, the three-dimensional data decoding device obtains information indicating the position information of the reference target of the attribute information of the processing object from the bitstream, and decodes the attribute information of the processing object with reference to the position information of the reference target indicated by the obtained information.

[0479] For example, the bitstream includes: a position parameter set (position PS) of control information including position information at each moment, and an attribute parameter set (attribute PS) of control information including attribute information at each moment. That is, the three-dimensional data decoding device decodes the position information at the processing target moment using the control information included in the position parameter set at the processing target moment, and decodes the attribute information at the processing target moment using the control information included in the attribute parameter set at the processing target moment.

[0480] For example, (attribute SPS). That is, the three-dimensional data decoding device decodes the position information at multiple moments using the control information included in the position sequence parameter set, and decodes the attribute information at multiple moments using the control information included in the attribute sequence parameter set.

[0481] For example, the bitstream includes: a stream parameter set (stream PS) of control information common to the position information at multiple moments and the attribute information at multiple moments. That is, the three-dimensional data decoding device decodes the position information at multiple moments and the attribute information at multiple moments using the control information included in the stream parameter set.

[0482] For example, the bitstream includes: an access unit header (AU header) of control information common within the access unit. That is, the three-dimensional data decoding device decodes the position information and attribute information included in the access unit using the control information included in the access unit header.

[0483] For example, the three-dimensional data decoding device independently decodes a group of pictures (GOP) composed of one or more access units. That is, the GOP is a random access unit.

[0484] For example, the bitstream includes: a GOP header of control information common within the GOP. That is, the three-dimensional data decoding device decodes the position information and attribute information included in the GOP using the control information included in the GOP header.

[0485] For example, the three-dimensional data decoding device includes a processor and a memory, and the processor uses the memory to perform the above processing.

[0486] As described above, the three-dimensional data encoding device and the three-dimensional data decoding device according to the embodiments of the present disclosure have been described, but the present disclosure is not limited to this embodiment.

[0487] In addition, each processing unit included in the three-dimensional data encoding device and the three-dimensional data decoding device according to the above embodiments can typically be implemented as a large-scale integration (LSI) of an integrated circuit. These can be made into one chip separately, or a part or all of them can be made into one chip.

[0488] Moreover, the integration circuit is not limited to LSI, and can also be implemented by dedicated circuits or general-purpose processors. It is also possible to use an FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconstruct the connections or settings of the circuit section inside the LSI.

[0489] Moreover, in each of the above-described embodiments, each component can be constituted by dedicated hardware, or can be implemented by executing a software program suitable for each component. Each component can also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0490] Moreover, the present disclosure can be implemented as a three-dimensional data encoding method or a three-dimensional data decoding method executed by a three-dimensional data encoding device and a three-dimensional data decoding device, etc.

[0491] Moreover, the division of the functional blocks in the block diagram is an example, and a plurality of functional blocks can be implemented as one functional block, one functional block can also be divided into a plurality of functional blocks, and a part of the function can also be moved to other functional blocks. Moreover, the functions of a plurality of functional blocks having similar functions can also be processed in parallel or time-division processed by a single piece of hardware or software.

[0492] Moreover, regarding the execution order of each step in the flowchart, it is an example given for specifically explaining the present disclosure, and it can also be an order other than the above. Moreover, a part of the above steps can also be executed simultaneously (in parallel) with other steps.

[0493] The three-dimensional data encoding device and the three-dimensional data decoding device, etc. of one or more forms have been described above based on the embodiments, but the present disclosure is not limited by these embodiments. Within the scope not departing from the gist of the present disclosure, forms obtained by performing various modifications conceivable by those skilled in the art on the present embodiment, and forms obtained by combining the components in different embodiments are all included within the scope of one or more forms.

[0494] Industrial Applicability

[0495] The present disclosure can be applied to a three-dimensional data encoding device and a three-dimensional data decoding device.

[0496] Explanation of the Drawing Texts

[0497] 4601 Three-Dimensional Data Encoding System

[0498] 4602 Three-Dimensional Data Decoding System

[0499] 4603 Sensor Terminal

[0500] 4604 External connection part

[0501] 4611 Point cloud data generation system

[0502] 4612 Prompt part

[0503] 4613 Encoding part

[0504] 4614 Multiplexing part

[0505] 4615 Input / output part

[0506] 4616 Control part

[0507] 4617 Sensor information acquisition part

[0508] 4618 Point cloud data generation part

[0509] 4621 Sensor information acquisition part

[0510] 4622 Input / output part

[0511] 4623 Demultiplexing part

[0512] 4624 Decoding part

[0513] 4625 Prompt part

[0514] 4626 User interface

[0515] 4627 Control part

[0516] 4630 First encoding part

[0517] 4631 Location information encoding part

[0518] 4632 Attribute information encoding part

[0519] 4633 Additional information encoding part

[0520] 4634 Multiplexing part

[0521] 4640 First decoding part

[0522] 4641 Demultiplexing part

[0523] 4642 Location information decoding part

[0524] 4643 Attribute information decoding part

[0525] 4644 Additional information decoding part

[0526] 4650 Second encoding part

[0527] 4651 Additional information generation part

[0528] 4652 Position Image Generation Unit

[0529] 4653 Attribute Image Generation Unit

[0530] 4654 Video Encoding Unit

[0531] 4655 Additional Information Encoding Unit

[0532] 4656 Multiplexing Unit

[0533] 4660 Second Decoding Unit

[0534] 4661 Demultiplexing Unit

[0535] 4662 Video Decoding Unit

[0536] 4663 Additional Information Decoding Unit

[0537] 4664 Position Information Generation Unit

[0538] 4665 Attribute Information Generation Unit

[0539] 4670 Encoding Unit

[0540] 4671 Multiplexing Unit

[0541] 4680 Decoding Unit

[0542] 4681 Demultiplexing Unit

[0543] 4710 First Multiplexing Unit

[0544] 4711 File Transformation Unit

[0545] 4720 First Demultiplexing Unit

[0546] 4721 File Inverse Transformation Unit

[0547] 4730 Second Multiplexing Unit

[0548] 4731 File Transformation Unit

[0549] 4740 Second Demultiplexing Unit

[0550] 4741 File Inverse Transformation Unit

[0551] 4750 Third Multiplexing Unit

[0552] 4751 File Transformation Unit

[0553] 4760 Third Demultiplexing Unit

[0554] 4761 File Inverse Transformation Unit

[0555] 4801 Encoding Unit

[0556] 4802 Multiplexing Unit

Claims

1. A three-dimensional data generation method, comprising: Generate a geometric information unit; Generate an attribute information unit; and Generate a unit containing a plurality of parameters for decoding the geometric information unit and the attribute information unit, wherein, Each of the geometric information unit, the attribute information unit, and the unit containing a plurality of parameters includes information indicating the type of data contained in the unit.

2. The method according to claim 1, wherein, The geometric information unit includes a first header and a first payload, The attribute information unit includes a second header and a second payload.

3. The method according to claim 1, wherein, The geometric information unit includes encoded geometric data, The attribute information unit includes encoded attribute data.

4. An encoded three-dimensional data processing method, comprising: Obtain a geometric information unit; Obtain an attribute information unit; and Obtain a unit containing a plurality of parameters for decoding the geometric information unit, wherein, Each of the geometric information unit, the attribute information unit, and the unit containing a plurality of parameters includes information indicating the type of data contained in the unit.

5. The method according to claim 4, wherein, The geometric information unit includes a first header and a first payload, and The attribute information unit includes a second header and a second payload.

6. The method according to claim 4, wherein, The geometric information unit includes encoded geometric data, and The attribute information unit includes encoded attribute data.

7. A three-dimensional data generation device, the device comprising: A processor; and A memory, wherein, Using the memory, the processor executes: Generate a geometric information unit; Generate an attribute information unit; and Generate a unit containing a plurality of parameters for decoding the geometric information unit and the attribute information unit, wherein, Each of the geometric information unit, the attribute information unit, and the unit containing a plurality of parameters includes information indicating the type of data contained in the unit.

8. An encoded three-dimensional data processing device, the device comprising: A processor; and A memory, wherein, Using the memory, the processor executes: Obtain a geometric information unit; Obtain an attribute information unit; and Obtain a unit containing a plurality of parameters for decoding the geometric information unit, wherein, Each of the geometric information unit, the attribute information unit, and the unit containing a plurality of parameters includes information indicating the type of data contained in the unit.

Images