Three-dimensional data storage method, three-dimensional data acquisition method, three-dimensional data storage device and three-dimensional data acquisition device

Abstract

A three-dimensional data storage method, obtaining one or more units storing a coded stream after encoding point group data (S4781), saving one or more units into a file (S4782), and in the saving (S4782), saving information indicating that the data stored in the file is data after encoding the point group data into the control information of the file. For example, the information can also indicate the encoding method used for encoding the point group data in the first encoding method and the second encoding method.

Description

Technical Field

[0001] The present disclosure relates to a three-dimensional data storage method, a three-dimensional data acquisition method, a three-dimensional data storage device, and a three-dimensional data acquisition device. Background Art

[0002] In the future, devices and services that make use of 3D data will become more common in large fields such as computer vision, map information, monitoring, infrastructure inspection, or image distribution for autonomous operation of cars or robots. 3D data is obtained by various methods such as distance sensors such as rangefinders, stereo cameras, or a combination of multiple single-lens cameras.

[0003] As a method of expressing three-dimensional data, there is a method called point cloud, which expresses the shape of a three-dimensional structure through a group of points in a three-dimensional space. The position and color of the point group are stored in the point cloud. Although point cloud is expected to become the mainstream method of expressing three-dimensional data, the amount of point group data is very large. Therefore, in the accumulation or transmission of three-dimensional data, it is necessary to compress the data volume through encoding, just like two-dimensional dynamic images (as an example, there are MPEG-4AVC or HEVC standardized by MPEG).

[0004] Furthermore, compression of point clouds is partially supported by a public library (PointCloud Library) that performs point cloud association processing.

[0005] Furthermore, there is a known technique for searching for facilities around a vehicle using three-dimensional map data and displaying the facilities (for example, refer to Patent Document 1).

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1 International Publication No. 2014 / 020663 Summary of the invention

[0009] Problems to be solved by the invention

[0010] In an apparatus for processing three-dimensional data, it is possible to reduce the amount of processing or increase the speed of processing.

[0011] The purpose of the present disclosure is to provide a three-dimensional data storage method, a three-dimensional data acquisition method, a three-dimensional data storage device, or a three-dimensional data acquisition device that can reduce the processing amount or speed up the processing in a device that processes three-dimensional data.

[0012] Means for solving problems

[0013] A three-dimensional data storage method in one form of the present invention obtains one or more units storing a coded stream after encoding point group data, and saves the one or more units in a file. During the saving, information indicating that the data stored in the file is data after encoding the point group data is saved in control information of the file.

[0014] A three-dimensional data acquisition method in one form of the present invention obtains a file storing one or more units, wherein the one or more units store a coded stream obtained by encoding point group data, and the one or more units are obtained from the file, wherein control information of the file includes information indicating that the data stored in the file is data obtained by encoding the point group data.

[0015] Effects of the Invention

[0016] The present disclosure can provide a three-dimensional data storage method, a three-dimensional data acquisition method, a three-dimensional data storage device, or a three-dimensional data acquisition device that can reduce the amount of processing or increase the speed of processing in a device that processes three-dimensional data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a diagram showing the structure of a three-dimensional data encoding and decoding system according to the first embodiment.

[0018] Figure 2 This is a diagram showing a structural example of point cloud data in Implementation Example 1.

[0019] Figure 3 This is a diagram showing a structural example of a data file that describes point group data information according to Implementation Example 1.

[0020] Figure 4 This is a diagram showing types of point cloud data in Implementation Example 1.

[0021] Figure 5 This is a diagram showing the structure of the first encoding unit according to implementation mode 1.

[0022] Figure 6 This is a block diagram of the first encoding unit according to implementation mode 1.

[0023] Figure 7 This is a diagram showing the structure of the first decoding unit according to Implementation Example 1.

[0024] Figure 8 This is a block diagram of the first decoding unit according to Implementation Example 1.

[0025] Fig. 9 This is a diagram showing the structure of the second encoding unit according to implementation mode 1.

[0026] Fig.10 This is a block diagram of the second encoding unit according to implementation mode 1.

[0027] Fig.11 This is a diagram showing the structure of the second decoding unit according to implementation mode 1.

[0028] Fig.12 This is a block diagram of the second decoding unit according to implementation mode 1.

[0029] Fig.13 This is a diagram showing a protocol stack related to PCC coded data according to the first embodiment.

[0030] Fig.14 This is a diagram showing a protocol stack according to the first embodiment.

[0031] Fig.15 This is a diagram showing a syntax example of a NAL unit according to the first embodiment.

[0032] Fig.16 This is a diagram showing a syntax example of a NAL unit header according to the first embodiment.

[0033] Fig.17 This is a diagram showing a semantic example of pcc_codec_type according to the first implementation mode.

[0034] Fig.18 This is a diagram showing a semantic example of pcc_nal_unit_type according to the first implementation mode.

[0035] Fig.19 This is a flowchart of the encoding process of implementation mode 1.

[0036] Fig. 20 This is a flowchart of the decoding process of the second decoding unit in implementation mode 1.

[0037] Fig.21 This is a flowchart of the decoding process of the first decoding unit in Implementation Example 1.

[0038] Fig. 22 This is a diagram showing a protocol stack according to the second implementation mode.

[0039] Fig.23 This is a diagram showing a syntax example of a NAL unit used in codec 2 according to the second embodiment.

[0040] Fig.24 This is a diagram showing a syntax example of a NAL unit header used by codec 2 according to the second embodiment.

[0041] Fig.25 This is a diagram showing a semantic example of codec2_nal_unit_type according to the second implementation mode.

[0042] Fig.26This is a diagram showing a syntax example of a NAL unit used in codec 1 according to the second embodiment.

[0043] Fig. 27 This is a diagram showing a syntax example of a NAL unit header used in codec 1 according to the second embodiment.

[0044] Fig.28 This is a diagram showing a semantic example of codec1_nal_unit_type according to the second implementation mode.

[0045] Fig.29 This is a flowchart of the encoding process of implementation mode 2.

[0046] Fig.30 This is a flowchart of the decoding process of implementation mode 2.

[0047] Fig.31 This is a diagram showing a protocol stack according to the third implementation mode.

[0048] Fig.32 This is a diagram showing a syntax example of a NAL unit according to the third embodiment.

[0049] Fig.33 This is a diagram showing a syntax example of a NAL unit header according to the third embodiment.

[0050] Fig.34 This is a diagram showing a semantic example of pcc_nal_unit_type according to the third implementation mode.

[0051] Fig.35 This is a flowchart of the encoding process of implementation mode 3.

[0052] Fig.36 This is a flowchart of the decoding process of implementation mode 3.

[0053] Fig.37 This is a flowchart of the encoding process of a modified example of the implementation method.

[0054] Fig.38 This is a flowchart of a decoding process according to a variation of the embodiment.

[0055] Fig.39 This is a block diagram of the encoding unit of implementation mode 4.

[0056] Fig.40 This is a block diagram of the decoding unit of implementation mode 4.

[0057] Fig.41 This is a flowchart of the encoding process of implementation mode 4.

[0058] Fig.42 This is a flowchart of the decoding process of implementation mode 4.

[0059] Fig.43 This is a diagram showing the basic structure of ISOBMFF related to implementation example 5.

[0060] Fig.44 This is a diagram showing a protocol stack related to implementation mode 5.

[0061] Fig.45 This is a diagram showing an example of storing NAL units in a file for codec 1 according to the fifth embodiment.

[0062] Fig.46 This is a diagram showing an example of storing the NAL unit related to Implementation Example 5 in a file for codec 2.

[0063] Fig.47 This is a diagram showing the structure of the first multiplexing unit related to Implementation Example 5.

[0064] Fig.48 This is a diagram showing the structure of the first inverse multiplexing unit related to implementation mode 5.

[0065] Fig.49 This is a diagram showing the structure of the second multiplexing unit related to Implementation Example 5.

[0066] Fig.50 This is a diagram showing the structure of the second inverse multiplexing unit related to implementation mode 5.

[0067] Fig.51 This is a flowchart of the processing performed by the first multiplexing unit in implementation mode 5.

[0068] Fig.52 This is a flowchart of the processing performed by the second multiplexing unit in implementation mode 5.

[0069] Fig.53 This is a flowchart of the processing performed by the first demultiplexing unit and the first decoding unit according to the fifth embodiment.

[0070] Fig.54 This is a flowchart showing the processing performed by the second demultiplexing unit and the second decoding unit related to implementation mode 5.

[0071] Fig.55 This is a diagram showing the structure of the encoding unit and the third multiplexing unit related to the sixth embodiment.

[0072] Fig.56 This is a diagram showing the structure of the third inverse multiplexing unit and decoding unit related to implementation mode 6.

[0073] Fig.57 This is a flowchart of the processing performed by the third multiplexing unit in implementation mode 6.

[0074] Fig.58This is a flowchart of the processing performed by the third inverse multiplexing unit and the decoding unit in accordance with the sixth embodiment.

[0075] Fig.59 This is a flowchart of the processing performed by the three-dimensional data storage device according to the sixth embodiment.

[0076] Fig.60 This is a flowchart of the processing performed by the three-dimensional data acquisition device according to the sixth embodiment. DETAILED DESCRIPTION

[0077] A three-dimensional data storage method in one form of the present disclosure obtains one or more units storing a coded stream after encoding point group data, and stores the one or more units in a file. During the storing, information indicating that the data stored in the file is data after encoding the point group data is saved in control information of the file.

[0078] Thus, in a device that processes a file generated by the three-dimensional data storage method, it is possible to refer to the control information of the file and determine at an early stage whether the data stored in the file is encoded data of point group data, thereby reducing the processing amount of the device or speeding up the processing.

[0079] For example, the information may further indicate a coding method used for coding the point cloud data, out of a first coding method and a second coding method.

[0080] Thus, in an apparatus that processes a file generated by the three-dimensional data storage method, the codec used for the data stored in the file can be determined at an early stage by referring to the file control information, thereby reducing the amount of processing of the apparatus or increasing the speed of processing.

[0081] For example, the first encoding method may be a method of encoding position information of the position of point group data represented by an N (N is an integer greater than 2) fork tree, and using the position information to encode attribute information, and the second encoding method may be a method of generating a two-dimensional image from the point group data, and encoding the two-dimensional image using an image encoding method.

[0082] For example, the file may comply with ISOBMFF (ISO based media file format).

[0083] A three-dimensional data acquisition method in one form of the present disclosure obtains a file storing one or more units of a coded stream obtained by encoding point group data, and obtains the one or more units from the file, wherein control information of the file includes information indicating that the data stored in the file is data obtained by encoding point group data.

[0084] Thus, the three-dimensional data acquisition method can determine at an early stage whether the data stored in the file is the coded data of the point group data. Thus, the processing amount of the device or the subsequent device performing the three-dimensional data acquisition method can be reduced or the processing speed can be increased.

[0085] For example, the information may further indicate a coding method used for the coding, out of a first coding method and a second coding method.

[0086] Thus, the three-dimensional data acquisition method can refer to the control information of the file and determine the codec used for the data stored in the file at an early stage. Thus, the processing amount of the device performing the three-dimensional data acquisition method or the subsequent device can be reduced or the processing speed can be increased.

[0087] For example, based on the information, the data encoded using any one encoding method may be obtained from the encoded point group data including the data encoded using the first encoding method and the data encoded using the second encoding method.

[0088] For example, the first encoding method may be a method of encoding position information of the position of point group data represented by an N (N is an integer greater than 2) fork tree, and using the position information to encode attribute information, and the second encoding method may be a method of generating a two-dimensional image from the point group data, and encoding the two-dimensional image using an image encoding method.

[0089] For example, the file may comply with ISOBMFF (ISO based media file format).

[0090] In addition, a three-dimensional data storage device in one form of the present invention comprises a processor and a memory, wherein the processor uses the memory to obtain one or more units storing a coded stream after encoding point group data, and saves the one or more units in a file, and during the saving, information indicating that the data stored in the file is data after encoding the point group data is saved in the control information of the file.

[0091] Thus, in a device that processes a file generated by the three-dimensional data storage method, it is possible to refer to the control information of the file and determine at an early stage whether the data stored in the file is encoded data of point group data, thereby reducing the processing amount of the device or speeding up the processing.

[0092] In addition, a three-dimensional data acquisition device in one form of the present invention comprises a processor and a memory, wherein the processor uses the memory to obtain a file storing one or more units of a coded stream after encoding point group data, and obtains the one or more units from the file, wherein control information of the file includes information indicating that the data stored in the file is data after encoding the point group data.

[0093] Thus, the three-dimensional data acquisition device can determine at an early stage whether the data stored in the file is the coded data of the point group data, thereby reducing the processing amount or speeding up the processing of the three-dimensional data acquisition device or the subsequent device.

[0094] In addition, these general or specific forms can be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, and can be implemented by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0095] The following detailed description of the implementation mode is given with reference to the accompanying drawings. In addition, the implementation modes to be described below are all specific examples of the present disclosure. The numerical values, shapes, materials, constituent elements, configuration positions of constituent elements, connection forms, steps, order of steps, etc. shown in the following implementation modes are all examples, and the main purpose is not to limit the present disclosure. Furthermore, the constituent elements of the following implementation modes that are not recorded in the technical solution showing the highest concept are described as arbitrary constituent elements.

[0096] (Implementation Method 1)

[0097] When point cloud coded data is used in actual devices or services, it is desirable to transmit and receive required information according to the application in order to reduce network bandwidth. However, such a function does not exist in the existing 3D data coding structure, and therefore there is no corresponding coding method.

[0098] In this embodiment, a three-dimensional data encoding method and a three-dimensional data encoding device are described for providing the function of sending and receiving required information according to the purpose in the encoded data of a three-dimensional point cloud, a three-dimensional data decoding method and a three-dimensional data decoding device are described for decoding the encoded data, a three-dimensional data multiplexing method for multiplexing the encoded data, and a three-dimensional data transmission method for transmitting the encoded data.

[0099] In particular, currently, the first encoding method and the second encoding method are studied as encoding methods (encoding methods) for point group data, but the structure of the encoded data and the method of saving the encoded data in the system format are not defined. In this case, there is a problem that MUX processing (multiplexing) in the encoding unit, or transmission or accumulation cannot be directly performed.

[0100] Furthermore, there is no method that supports a format in which two codecs, namely the first encoding method and the second encoding method, exist in a mixed form, such as PCC (Point Cloud Compression).

[0101] In this embodiment, a structure of PCC coded data in which two codecs, namely a first coding method and a second coding method, are mixed and a method of storing the coded data in a system format is described.

[0102] First, the configuration of the three-dimensional data (point cloud data) encoding and decoding system according to the present embodiment will be described. Figure 1 2 is a diagram showing a configuration example of a three-dimensional data encoding and decoding system according to the present embodiment. Figure 1 As shown, the three-dimensional data encoding and decoding system includes a three-dimensional data encoding system 4601, a three-dimensional data decoding system 4602, a sensor terminal 4603 and an external connection unit 4604.

[0103] The three-dimensional data encoding system 4601 generates encoded data or multiplexed data by encoding point group data as three-dimensional data. In addition, the three-dimensional data encoding system 4601 can be a three-dimensional data encoding device implemented by a single device, or a system implemented by multiple devices. In addition, the three-dimensional data encoding device can also be included in a part of the multiple processing units included in the three-dimensional data encoding system 4601.

[0104] The three-dimensional data encoding system 4601 includes a point cloud data generating system 4611, a presentation unit 4612, an encoding unit 4613, a multiplexing unit 4614, an input / output unit 4615, and a control unit 4616. The point cloud data generating system 4611 includes a sensor information obtaining unit 4617 and a point cloud data generating unit 4618.

[0105] The sensor information acquisition unit 4617 acquires sensor information from the sensor terminal 4603 and outputs the sensor information to the point cloud data generation unit 4618. The point cloud data generation unit 4618 generates point cloud data based on the sensor information and outputs the point cloud data to the encoding unit 4613.

[0106] The presentation unit 4612 presents the sensor information or point group data to the user. For example, the presentation unit 4612 displays information or an image based on the sensor information or point group data.

[0107] The encoding unit 4613 encodes (compresses) the point cloud data, and outputs the obtained encoded data, control information obtained in the encoding process, and other additional information to the multiplexing unit 4614. The additional information includes, for example, sensor information.

[0108] The multiplexing unit 4614 generates multiplexed data by multiplexing the coded data, control information, and additional information input from the coding unit 4613. The format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.

[0109] The input / output unit 4615 (e.g., a communication unit or an interface) outputs the multiplexed data to the outside. Alternatively, the multiplexed data is stored in a storage unit such as an internal memory. The control unit 4616 (or an application execution unit) controls each processing unit. That is, the control unit 4616 performs control such as encoding and multiplexing.

[0110] Furthermore, the sensor information may be input to the encoding unit 4613 or the multiplexing unit 4614. Furthermore, the input / output unit 4615 may directly output the point cloud data or the encoded data to the outside.

[0111] The transmission signal (multiplexed data) output from the three-dimensional data encoding system 4601 is input to the three-dimensional data decoding system 4602 via the external connection unit 4604 .

[0112] The three-dimensional data decoding system 4602 generates point group data as three-dimensional data by decoding the coded data or the multiplexed data. In addition, the three-dimensional data decoding system 4602 may be a three-dimensional data decoding device implemented by a single device, or may be a system implemented by multiple devices. In addition, the three-dimensional data decoding device may also include a part of the multiple processing units included in the three-dimensional data decoding system 4602.

[0113] The three-dimensional data decoding system 4602 includes a sensor information acquisition unit 4621 , an input / output unit 4622 , an inverse multiplexing unit 4623 , a decoding unit 4624 , a prompting unit 4625 , a user interface 4626 , and a control unit 4627 .

[0114] The sensor information acquisition unit 4621 acquires sensor information from the sensor terminal 4603 .

[0115] The input / output unit 4622 obtains the transmission signal, decodes the multiplexed data (file format or packet) according to the transmission signal, and outputs the multiplexed data to the demultiplexing unit 4623 .

[0116] The inverse multiplexing unit 4623 obtains the encoded data, control information, and additional information from the multiplexed data, and outputs the encoded data, control information, and additional information to the decoding unit 4624 .

[0117] The decoding unit 4624 reconstructs the point cloud data by decoding the encoded data.

[0118] The prompt unit 4625 prompts the user with the point group data. For example, the prompt unit 4625 displays information or images based on the point group data. The user interface 4626 obtains instructions based on the user's operation. The control unit 4627 (or the application execution unit) controls each processing unit. That is, the control unit 4627 controls demultiplexing, decoding, prompting, etc.

[0119] In addition, the input / output unit 4622 may also directly obtain point group data or coded data from the outside. In addition, the prompt unit 4625 may also obtain additional information such as sensor information and prompt information based on the additional information. In addition, the prompt unit 4625 may also perform prompts based on the user's instructions obtained by the user interface 4626.

[0120] The sensor terminal 4603 generates sensor information, which is information obtained by the sensor. The sensor terminal 4603 is a terminal equipped with a sensor or a camera, for example, a moving object such as a car, a flying object such as an airplane, a mobile terminal or a camera.

[0121] The sensor information that can be obtained by the sensor terminal 4603 is, for example, (1) the distance between the sensor terminal 4603 and the object, or the reflectivity of the object, obtained by LIDAR, millimeter wave radar, or infrared sensor, (2) the distance between the camera and the object, or the reflectivity of the object, obtained from multiple monocular camera images or stereo camera images. In addition, the sensor information may also include the posture, orientation, rotation (angular velocity), position (GPS information or altitude), speed or acceleration of the sensor. In addition, the sensor information may also include temperature, air pressure, humidity, or magnetism.

[0122] The external connection unit 4604 is implemented by an integrated circuit (LSI or IC), an external storage unit, communication with a cloud server via the Internet, or broadcasting.

[0123] Next, point group data will be described. Figure 2 It is a diagram showing the structure of point group data. Figure 3 It is a diagram showing a structural example of a data file that describes information on point group data.

[0124] Point group data includes data of multiple points. The data of each point includes position information (three-dimensional coordinates) and attribute information corresponding to the position information. A group of multiple such points is called a point group. For example, a point group represents the three-dimensional shape of an object.

[0125] Sometimes, position information (Position) such as three-dimensional coordinates is also called geometry (Geometry). In addition, the data of each point can also include attribute information (attribute) of multiple attribute categories. Attribute categories are, for example, color or reflectivity.

[0126] One piece of attribute information may be associated with one piece of location information, or a plurality of pieces of attribute information having different attribute categories may be associated with one piece of location information. In addition, a plurality of pieces of attribute information having the same attribute category may be associated with one piece of location information.

[0127] Figure 3 The illustrated data file structure example is an example of a case where position information and attribute information correspond to each other on a one-to-one basis, and indicates position information and attribute information of N points constituting point cloud data.

[0128] The position information is, for example, information on three axes, x, y, and z. The attribute information is, for example, color information of RGB. Representative data files include ply files and the like.

[0129] Next, types of point cloud data will be described. Figure 4 is a graph showing the types of point group data. Figure 4 As shown, the point group data includes static objects and dynamic objects.

[0130] A static object is a 3D point group data at any time (a certain moment). A dynamic object is a 3D point group data that changes over time. Hereinafter, the 3D point group data at a certain moment is referred to as a PCC frame or frame.

[0131] The object may be a point group whose area is limited to a certain extent, such as normal image data, or a large-scale point group whose area is not limited, such as map information.

[0132] In addition, there are point group data of various densities, and there may be sparse point group data and dense point group data.

[0133] The details of each processing unit are described below. The sensor information is obtained by various methods such as a distance sensor such as LIDAR or a rangefinder, a stereo camera, or a combination of multiple monocular cameras. The point group data generation unit 4618 generates point group data based on the sensor information obtained by the sensor information acquisition unit 4617. The point group data generation unit 4618 generates position information as point group data and adds attribute information for the position information to the position information.

[0134] The point group data generation unit 4618 may also process the point group data when generating the position information or the additional attribute information. For example, the point group data generation unit 4618 may also reduce the amount of data by deleting point groups with repeated positions. In addition, the point group data generation unit 4618 may also transform the position information (position conversion, rotation or standardization, etc.) and may also render the attribute information.

[0135] In addition, Figure 1 In the figure, the point group data generating system 4611 is included in the three-dimensional data encoding system 4601, but can also be independently set outside the three-dimensional data encoding system 4601.

[0136] The encoding unit 4613 encodes the point cloud data based on a predetermined encoding method, thereby generating encoded data. There are generally two types of encoding methods. The first type is an encoding method using position information, which is hereinafter referred to as the first encoding method. The second type is an encoding method using a video codec, which is hereinafter referred to as the second storage method.

[0137] The decoding unit 4624 decodes the encoded data based on a predetermined encoding method, thereby decoding the point cloud data.

[0138] The multiplexing unit 4614 generates multiplexed data by multiplexing the coded data using an existing multiplexing method. The generated multiplexed data is transmitted or stored. In addition to the PCC coded data, the multiplexing unit 4614 also multiplexes other media such as images, sounds, subtitles, applications, files, or reference time information. In addition, the multiplexing unit 4614 can also multiplex attribute information associated with sensor information or point group data.

[0139] As multiplexing methods or file formats, there are ISOBMFF, MPEG-DASH which is a transmission method based on ISOBMFF, MMT, MPEG-2TS Systems, RMP, etc.

[0140] The demultiplexing unit 4623 extracts PCC coded data, other media, time information, etc. from the multiplexed data.

[0141] The input / output unit 4615 transmits the multiplexed data using a method consistent with a transmission medium or storage medium such as broadcasting or communication. The input / output unit 4615 can communicate with other devices via the Internet, or can communicate with a storage unit such as a cloud server.

[0142] As the communication protocol, http, ftp, TCP, UDP, etc. can be used. Either a PULL type communication method or a PUSH type communication method can be used.

[0143] Any of wired transmission and wireless transmission can be used. As wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), or coaxial cable, etc. are used. As wireless transmission, wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), or millimeter wave, etc. are used.

[0144] In addition, as a broadcasting method, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 is used.

[0145] Figure 5 This diagram shows a structure of a first coding unit 4630 which is an example of a coding unit 4613 that performs coding using the first coding method. Figure 6 4 is a block diagram of the first coding unit 4630. The first coding unit 4630 generates coded data (coded stream) by coding the point cloud data using the first coding method. The first coding unit 4630 includes a position information coding unit 4631, an attribute information coding unit 4632, an additional information coding unit 4633, and a multiplexing unit 4634.

[0146] The first coding unit 4630 is characterized in that the coding is performed with awareness of the three-dimensional structure. In addition, the first coding unit 4630 is characterized in that the attribute information coding unit 4632 performs coding using information obtained from the position information coding unit 4631. The first coding method is also called GPCC (Geometry based PCC).

[0147] The point group data is PCC point group data such as a PLY file, or PCC point group data generated based on sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData). The position information is input to the position information encoding unit 4631, the attribute information is input to the attribute information encoding unit 4632, and the additional information is input to the additional information encoding unit 4633.

[0148] The position information encoding unit 4631 generates encoded position information (Compressed Geometry) as encoded data by encoding the position information. For example, the position information encoding unit 4631 uses an N-ary tree structure such as an octree to encode the position information. Specifically, in the octree, the object space is divided into 8 nodes (subspaces), and 8 bits of information (occupancy code) are generated to indicate whether each node contains a point group. In addition, the node containing the point group is further divided into 8 nodes, and 8 bits of information are generated to indicate whether each of the 8 nodes contains a point group. This process is repeated until it becomes below the threshold of the number of point groups contained in a predetermined layer or node.

[0149] The attribute information encoding unit 4632 generates the encoded attribute information (Compressed Attribute) as the encoded data by encoding using the structure information generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 determines the reference point (reference node) to be referred to in the encoding of the object point (object node) of the processing object based on the octree structure generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 refers to a node whose parent node in the octree is the same as the parent node of the object node among the surrounding nodes or adjacent nodes. In addition, the method of determining the reference relationship is not limited to this.

[0150] In addition, the encoding process of the attribute information may include at least one of a quantization process, a prediction process, and an arithmetic coding process. In this case, the reference means using the reference node in the calculation of the predicted value of the attribute information, or using the state of the reference node in the determination of the encoding parameter (for example, indicating whether the occupancy information of the point group is included in the reference node). For example, the encoding parameter is a quantization parameter in the quantization process, or a context in the arithmetic coding, etc.

[0151] The additional information encoding unit 4633 generates encoded additional information (Compressed MetaData) as encoded data by encoding compressible data in the additional information.

[0152] The multiplexing unit 4634 generates a coded stream (Compressed Stream) as coded data by multiplexing the coding position information, the coding attribute information, the coding additional information, and other additional information. The generated coded stream is output to a processing unit of the system layer (not shown).

[0153] Next, the first decoding unit 4640 which is an example of the decoding unit 4624 that performs decoding according to the first encoding method is described. Figure 7 This is a diagram showing the structure of the first decoding unit 4640. Figure 8 46 is a block diagram of a first decoding unit 4640. The first decoding unit 4640 generates point cloud data by decoding coded data (coded stream) coded by the first coding method by the first coding method. The first decoding unit 4640 includes a demultiplexing unit 4641, a position information decoding unit 4642, an attribute information decoding unit 4643, and an additional information decoding unit 4644.

[0154] A coded stream (Compressed Stream) as coded data is input to the first decoding unit 4640 from a processing unit of a system layer (not shown).

[0155] The inverse multiplexing unit 4641 separates the encoded position information (Compressed Geometry), the encoded attribute information (Compressed Attribute), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0156] The position information decoding unit 4642 generates position information by decoding the encoded position information. For example, the position information decoding unit 4642 restores the position information of the point group represented by three-dimensional coordinates based on the encoded position information represented by an N-ary tree structure such as an octree.

[0157] The attribute information decoding unit 4643 decodes the encoded attribute information based on the structure information generated by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 determines a reference point (reference node) to be referred to in decoding the object point (object node) of the processing object based on the octree structure obtained by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 refers to a node whose parent node in the octree is the same as the parent node of the object node among the surrounding nodes or adjacent nodes. In addition, the method of determining the reference relationship is not limited to this.

[0158] In addition, the decoding process of the attribute information may also include at least one of an inverse quantization process, a prediction process, and an arithmetic decoding process. In this case, the reference means that the reference node is used in the calculation of the predicted value of the attribute information, or the state of the reference node is used in the determination of the decoded parameter (for example, indicating whether the occupancy information of the point group is included in the reference node). For example, the decoded parameter is a quantization parameter in the inverse quantization process, or a context in the arithmetic decoding, etc.

[0159] The additional information decoding unit 4644 generates additional information by decoding the encoded additional information. In addition, the first decoding unit 4640 uses the additional information required for decoding processing of the position information and the attribute information during decoding, and outputs the additional information required by the application to the outside.

[0160] Next, the second encoding unit 4650 which is an example of the encoding unit 4613 that performs encoding using the second encoding method is described. Fig. 9 This is a diagram showing the structure of the second encoding unit 4650. Fig.10 This is a block diagram of the second encoding unit 4650.

[0161] The second coding unit 4650 generates coded data (coded stream) by coding the point cloud data using the second coding method. The second coding unit 4650 includes an additional information generating unit 4651 , a position image generating unit 4652 , an attribute image generating unit 4653 , a video coding unit 4654 , an additional information coding unit 4655 , and a multiplexing unit 4656 .

[0162] The second coding unit 4650 generates a position image and an attribute image by projecting a three-dimensional structure onto a two-dimensional image, and codes the generated position image and attribute image using an existing video coding method. The second coding method is also called VPCC (video based PCC).

[0163] The point group data is PCC point group data such as a PLY file or PCC point group data generated based on sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData).

[0164] The additional information generating unit 4651 generates mapping information of a plurality of two-dimensional images by projecting a three-dimensional structure onto a two-dimensional image.

[0165] The position image generation unit 4652 generates a position image (Geometry Image) based on the position information and the mapping information generated by the additional information generation unit 4651. The position image is, for example, a distance image representing the distance (Depth) as a pixel value. In addition, the distance image can be an image of multiple point groups observed from one viewpoint (an image of multiple point groups projected on one two-dimensional plane), or multiple images of multiple point groups observed from multiple viewpoints, or a single image formed by merging these multiple images.

[0166] The attribute image generation unit 4653 generates an attribute image based on the attribute information and the mapping information generated by the additional information generation unit 4651. The attribute image is, for example, an image that represents the attribute information (e.g., color (RGB)) as a pixel value. In addition, the image may be an image of a plurality of point groups observed from a single viewpoint (an image in which a plurality of point groups are projected on a single two-dimensional plane), or may be a plurality of images of a plurality of point groups observed from a plurality of viewpoints, or may be a single image formed by merging these plurality of images.

[0167] The image encoding unit 4654 encodes the position image and the attribute image using an image encoding method, thereby generating a compressed position image (Compressed Geometry Image) and a compressed attribute image (Compressed Attribute Image) as encoded data. In addition, as the image encoding method, any known encoding method can be used. For example, the image encoding method is AVC or HEVC.

[0168] The additional information encoding unit 4655 generates coded additional information (Compressed MetaData) by encoding the additional information and mapping information included in the point cloud data.

[0169] The multiplexing unit 4656 generates a coded stream (Compressed Stream) as coded data by multiplexing the coding position image, the coding attribute image, the coding additional information, and other additional information. The generated coded stream is output to a processing unit of the system layer (not shown).

[0170] Next, the second decoding unit 4660 which is an example of the decoding unit 4624 that performs decoding according to the second encoding method is described. Fig.11 This is a diagram showing the structure of the second decoding unit 4660. Fig.12 46 is a block diagram of a second decoding unit 4660. The second decoding unit 4660 generates point cloud data by decoding coded data (coded stream) coded by the second coding method by the second coding method. The second decoding unit 4660 includes an inverse multiplexing unit 4661, a video decoding unit 4662, an additional information decoding unit 4663, a position information generating unit 4664, and an attribute information generating unit 4665.

[0171] A coded stream (Compressed Stream) as coded data is input to the second decoding unit 4660 from a processing unit of a system layer (not shown).

[0172] The inverse multiplexing unit 4661 separates the coded position image (Compressed Geometry Image), the coded attribute image (Compressed Attribute Image), the coded additional information (Compressed MetaData), and other additional information from the coded data.

[0173] The video decoding unit 4662 generates a position image and an attribute image by decoding the encoded position image and the encoded attribute image using a video encoding method. In addition, as the video encoding method, any known encoding method can be used. For example, the video encoding method is AVC or HEVC.

[0174] The additional information decoding unit 4663 generates additional information including mapping information and the like by decoding the encoded additional information.

[0175] The position information generating unit 4664 generates the position information using the position image and the mapping information. The attribute information generating unit 4665 generates the attribute information using the attribute image and the mapping information.

[0176] The second decoding unit 4660 uses the additional information required for decoding during decoding, and outputs the additional information required by the application to the outside.

[0177] The following describes the issues in the PCC coding method. Fig.13 It is a diagram showing a protocol stack related to PCC coded data. Fig.13 This shows an example of multiplexing, transmitting, or accumulating data of other media such as images (for example, HEVC) or audio into PCC coded data.

[0178] The multiplexing method and file format have functions for multiplexing, transmitting or accumulating various coded data. In order to transmit or accumulate coded data, the coded data must be converted into a format of the multiplexing method. For example, HEVC stipulates a technology for storing coded data in a data structure called a NAL unit and storing the NAL unit in ISOBMFF.

[0179] On the other hand, currently, as coding methods for point group data, the first coding method (Codec1) and the second coding method (Codec2) are being studied, but the structure of the coded data and the method of saving the coded data in the system format are not defined, and there is a problem that MUX processing (multiplexing), transmission and accumulation in the coding unit cannot be directly performed.

[0180] In the following, unless a specific encoding method is mentioned, either the first encoding method or the second encoding method is indicated.

[0181] The following is an explanation of the method for defining the NAL unit of the present embodiment. For example, in previous codecs such as HEVC, a NAL unit of one format is defined for one codec. However, there is no method to support a mixed format of two codecs (hereinafter referred to as PCC codecs) such as the first coding method and the second coding method, as in PCC.

[0182] In this embodiment, a format common to the PCC codec is defined as a NAL unit, and an identifier of the NAL unit that depends on the PCC codec is further defined. Fig.14 This is a diagram showing a protocol stack in this case. Figures 15 to 17 This is a diagram showing an example of a NAL unit format common to codecs. Fig.15 This is a diagram showing a syntax example of a common PCC NAL unit (Common PCC NALUnit). Fig.16 This is a diagram showing a syntax example of a common PCC NAL unit header (Common PCC NAL Unit Header). Fig.17 This is a diagram showing a semantic example of pcc_codec_type. Fig.18This diagram shows an example of codec-dependent NAL unit type definition and a semantic example of pcc_nal_unit_type.

[0183] As the NAL unit format, a NAL unit format common to the PCC codec is defined. The NAL unit (pcc_nal_unit) includes a header (pcc_nal_unit_header), a payload (pcc_nal_unit_payload), and trailing bits (trailing_bits). Even when storing data of a codec of either the first encoding method or the second encoding method, the same format is used.

[0184] The NAL unit header (pcc_nal_unit_header) stores a codec type (pcc_codec_type) and a NAL unit type (pcc_nal_unit_type). The codec type indicates whether the PCC codec of the coded data stored in the NAL unit is the first coding method or the second coding method.

[0185] The NAL unit type indicates the type of the NAL unit that depends on the codec, and the type is defined for each codec. When the codec type is the first encoding method, the NAL unit type indicates the NAL unit type defined for the first encoding method. When the codec type is the second encoding method, the NAL unit type indicates the NAL unit type defined for the second encoding method. That is, the same value corresponds to different meanings for the NAL unit type defined for the first encoding method and the NAL unit type defined for the second encoding method.

[0186] In addition, in the header, the function of the codec type can also be incorporated into the NAL unit type. For example, a part of the information of the NAL unit type can also be used to indicate the codec type.

[0187] Next, the encoding process of this embodiment is described. Fig.19 4613 is a flowchart of the encoding process of this embodiment. The process in this figure shows the process of the first encoding unit 4630 or the second encoding unit 4650 when the above definition is used. In addition, the first encoding unit 4630 or the second encoding unit 4650 is not distinguished below and is recorded as the encoding unit 4613. In addition, the process in this figure is mainly composed of Figure 6 The multiplexing unit 4634 or Fig.10 The multiplexing unit 4656 shown is performed.

[0188] The process in the figure shows an example of encoding PCC data using either the first encoding method or the second encoding method, and it is known which PCC codec is used for encoding. For example, which PCC codec is used may be specified by a user or an external device.

[0189] First, the encoding unit 4613 encodes the PCC data using any codec of the first encoding method and the second encoding method (S4601).

[0190] When the codec used is the second encoding method (the second encoding method in S4602), the encoding unit 4613 sets the pcc_codec_type contained in the NAL unit header to a value indicating that the data contained in the payload of the NAL unit is data encoded using the second encoding method (S4603). In addition, the encoding unit 4613 sets the identifier of the NAL unit used for the second encoding method to the pcc_nal_unit_type in the NAL unit header (S4604). Then, the encoding unit 4613 generates a NAL unit having the set NAL unit header and containing encoded data in the payload. Then, the encoding unit 4613 sends the generated NAL unit (S4605).

[0191] On the other hand, when the codec used is the first encoding method (the first encoding method in S4602), the encoding unit 4613 sets the pcc_codec_type in the NAL unit header to a value indicating that the data contained in the payload of the NAL unit is data encoded using the first encoding method (S4606). In addition, the encoding unit 4613 sets the identifier of the NAL unit used for the first encoding method in the pcc_nal_unit_type in the NAL unit header (S4607). Then, the encoding unit 4613 generates a NAL unit having the set NAL unit header and containing encoded data in the payload. Then, the encoding unit 4613 sends the generated NAL unit (S4605).

[0192] Furthermore, in steps S4603 and S4606, when the function of pcc_code_type is included in pcc_nal_unit_type, the encoder 4613 may indicate in pcc_nal_unit_type whether the NAL unit is encoded using the first encoding method or the second encoding method.

[0193] Next, the decoding processing performed by the first decoding unit 4640 and the second decoding unit 4660 according to this embodiment will be described. Fig. 20 4660 is a flowchart showing the decoding process of the second decoding unit 4660. In addition, the process of this figure is mainly composed of Fig.12 The inverse multiplexing unit 4661 shown is performed.

[0194] In addition, the processing of this figure shows an example of encoding PCC data using either the second encoding method or the first encoding method. In addition, in this method, the inverse multiplexing unit 4661 included in the second decoding unit 4660 can refer to the information included in the NAL unit header to identify the codec type of the NAL unit. Therefore, the inverse multiplexing unit 4661 can output the required information to the video decoding unit 4662 according to the codec type.

[0195] First, the second decoding unit 4660 receives a NAL unit (S4611). For example, this NAL unit is generated in the process in the above-mentioned encoding unit 4613. That is, the header of this NAL unit includes pcc_codec_type and pcc_nal_unit_type.

[0196] Next, the second decoding unit 4660 determines whether pcc_codec_type included in the NAL unit header indicates the first encoding method or the second encoding method ( S4612 ).

[0197] When pcc_codec_type indicates the second coding method (the second coding method in S4612), the second decoding unit 4660 determines that the data included in the payload of the NAL unit is data encoded using the second coding method (S4613). Then, the second decoding unit 4660 identifies the data by using pcc_nal_unit_type included in the NAL unit header as an identifier of the NAL unit for the second coding method (S4614). Then, the second decoding unit 4660 decodes the PCC data using the decoding process of the second coding method (S4615).

[0198] On the other hand, when pcc_codec_type indicates the first coding method (the first coding method in S4612), the second decoding unit 4660 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4616). In this case, the second decoding unit 4660 does not process the NAL unit (S4617).

[0199] Furthermore, in step S4612, when the function of pcc_code_type is included in pcc_nal_unit_type, the second decoding unit 4660 may refer to pcc_nal_unit_type to determine whether the codec used in the data included in the NAL unit is the first encoding method or the second encoding method.

[0200] Fig.21 4640 is a flowchart showing the decoding process of the first decoding unit 4640. In addition, the process of this figure is mainly composed of Figure 8 The inverse multiplexing unit 4641 shown is performed.

[0201] In addition, the processing of the figure shows an example of encoding PCC data by either the first encoding method or the second encoding method. In addition, in this method, the inverse multiplexing unit 4641 included in the first decoding unit 4640 can refer to the information included in the NAL unit header to identify the codec type of the NAL unit. Therefore, the inverse multiplexing unit 4641 can output the required information to the position information decoding unit 4642 and the attribute information decoding unit 4643 according to the codec type.

[0202] First, the first decoding unit 4640 receives a NAL unit (S4621). For example, this NAL unit is generated in the process in the above-mentioned encoding unit 4613. That is, the header of this NAL unit includes pcc_codec_type and pcc_nal_unit_type.

[0203] Next, the first decoding unit 4640 determines whether pcc_codec_type included in the NAL unit header indicates the first encoding method or the second encoding method ( S4622 ).

[0204] When pcc_codec_type indicates the second coding method (the second coding method in S4622), the first decoding unit 4640 determines that the data included in the payload of the NAL unit is data encoded using the second coding method (S4623). In this case, the first decoding unit 4640 does not process the NAL unit (S4624).

[0205] On the other hand, when pcc_codec_type indicates the first coding method (the first coding method in S4622), the first decoding unit 4640 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4625). Then, the first decoding unit 4640 identifies the data by using pcc_nal_unit_type included in the NAL unit header as an identifier of the NAL unit for the first coding method (S4626). Then, the first decoding unit 4640 decodes the PCC data using the decoding process of the first coding method (S4627).

[0206] (Implementation Method 2)

[0207] In this embodiment, another method of defining a NAL unit is described. In this embodiment, a different format is defined for each PCC codec as a NAL unit. Furthermore, an identifier of a NAL unit is independently defined for each PCC codec.

[0208] Fig. 22 This is a diagram showing a protocol stack in this case. Fig.23 This is a diagram showing a syntax example of a NAL unit for codec 2 (codec2_nal_unit). Fig.24 This is a diagram showing a syntax example of a NAL unit header for codec 2 (codec2_nal_unit_header). Fig.25 This is a diagram showing an example of the semantics of codec2_nal_unit_type.

[0209] Fig.26 This is a diagram showing a syntax example of a NAL unit for codec 1 (codec1_nal_unit). Fig. 27 This is a diagram showing a syntax example of a NAL unit header for codec 1 (codec1_nal_unit_header). Fig.28 This is a diagram showing a semantic example of codec1_nal_unit_type.

[0210] As a NAL unit format, the NAL unit format is defined independently for each PCC codec. The NAL unit (codec1_nal_unit, codec2_nal_unit) includes a header (codec1_nal_unit_header, codec2_nal_unit_header), a payload (codec1_nal_unit_payload, codec2_nal_unit_payload), and a trailing bit (trailing_bits). The NAL unit (codec1_nal_unit) used for the first encoding method and the NAL unit (codec2_nal_unit) used for the second encoding method can have the same structure or different structures. The size of the NAL unit used for the first encoding method and the size of the NAL unit used for the second encoding method can also be different.

[0211] The data encoded by the first encoding method is stored in the NAL unit for the first encoding method. The data encoded by the second encoding method is stored in the NAL unit for the second encoding method.

[0212] The NAL unit header (codec1_nal_unit_header, codec2_nal_unit_header) stores the NAL unit type (codec1_nal_unit_type, codec2_nal_unit_type). The NAL unit type is independent for each codec, and the type is defined for each codec. That is, in the NAL unit used for the first encoding method, the NAL unit type defined for the first encoding method is recorded. In the NAL unit used for the second encoding method, the NAL unit type defined for the second encoding method is recorded.

[0213] By adopting this method, the first encoding method and the second encoding method can be handled as different codecs.

[0214] Next, the encoding process of this embodiment is described. Fig.29 46 is a flowchart of the encoding process of this embodiment. The process in this figure shows the process of the first encoding unit 4630 or the second encoding unit 4650 when the above definition is used. In addition, the process in this figure is mainly composed of Figure 6 The multiplexing unit 4634 or Fig.10 The multiplexing unit 4656 shown is performed.

[0215] The process in the figure shows an example of encoding PCC data using either the first encoding method or the second encoding method, and it is known which PCC codec is used for encoding. For example, which PCC codec is used may be specified by a user or an external device.

[0216] First, the encoding unit 4613 encodes the PCC data using any codec of the first encoding method and the second encoding method (S4631).

[0217] When the codec used is the second encoding method (the second encoding method in S4632), the encoding unit 4613 generates a NAL unit in the NAL unit format for the second encoding method (S4633). Next, the encoding unit 4613 sets the identifier of the NAL unit for the second encoding method to codec2_nal_unit_type included in the NAL unit header (S4634). Then, the encoding unit 4613 generates a NAL unit having the set NAL unit header and including the encoded data in the payload. Then, the encoding unit 4613 sends the generated NAL unit (S4635).

[0218] On the other hand, when the codec used is the first encoding method (the first encoding method in S4632), the encoding unit 4613 generates a NAL unit in the NAL unit format for the first encoding method (S4636). Next, the encoding unit 4613 sets the identifier of the NAL unit for the first encoding method in codec1_nal_unit_type in the NAL unit header (S4637). Then, the encoding unit 4613 generates a NAL unit having the set NAL unit header and including encoded data in the payload. Then, the encoding unit 4613 sends the generated NAL unit (S4635).

[0219] Next, the decoding process of this embodiment is described. Fig.30 4624. The process of this figure shows the process of the first decoding unit 4640 or the second decoding unit 4660 when the above definition is used. In addition, the first decoding unit 4640 or the second decoding unit 4660 is not distinguished below and is recorded as the decoding unit 4624. In addition, the process of this figure is mainly composed of Figure 8 The inverse multiplexing unit 4641 or Fig.12 The inverse multiplexing unit 4661 shown is performed.

[0220] In addition, the processing of this figure shows an example of encoding PCC data by any one of the first encoding method and the second encoding method, and it is assumed that it is known in which PCC codec the data is encoded. For example, information indicating the codec used is included in the transmission signal, multiplexed data, or encoded data, and the decoding unit 4624 refers to the information to determine the codec used. In addition, the decoding unit 4624 can also determine the codec used based on a signal obtained separately from these signals.

[0221] When the codec used is the second coding method (the second coding method in S4641), the decoding unit 4624 receives the NAL unit in the format for the second coding method (S4642). Next, assuming that the NAL unit is for the second coding method, the decoding unit 4624 identifies the data using the NAL unit format for the second coding method and codec2_nal_unit_type for the second coding method (S4643). Next, the decoding unit 4624 decodes the PCC data using the decoding process of the second coding method (S4644).

[0222] On the other hand, when the codec used is the first coding method (the first coding method in S4641), the decoding unit 4624 receives the NAL unit in the format for the first coding method (S4645). Next, assuming that the NAL unit is for the first coding method, the decoding unit 4624 identifies the data using the NAL unit format for the first coding method and codec1_nal_unit_type for the first coding method (S4646). Next, the decoding unit 4624 decodes the PCC data using the decoding process of the first coding method (S4747).

[0223] (Implementation method 3)

[0224] In this embodiment, another method of defining a NAL unit is described. In this embodiment, a format common to the PCC codec is defined as a NAL unit. In addition, an identifier of a NAL unit common to the PCC codec is defined.

[0225] Fig.31 This is a diagram showing a protocol stack in this case. Figure 32 to Figure 34 This is a diagram showing an example of a NAL unit format common to codecs. Fig.32 This is a diagram showing a syntax example of a common PCC NAL unit (Common PCC NAL Unit). Fig.33 This is a diagram showing a syntax example of a common PCC NAL unit header (Common PCC NAL Unit Header). Fig.34 This is a diagram showing a semantic example of pcc_codec_type.

[0226] As the NAL unit format, a NAL unit format common to the PCC codec is defined. The NAL unit (pcc_nal_unit) includes a header (pcc_nal_unit_header), a payload (pcc_nal_unit_payload), and trailing bits (trailing_bits). The same format is used even when storing data of any codec in the first encoding method and the second encoding method.

[0227] The NAL unit header (pcc_nal_unit_header) stores the NAL unit type (pcc_nal_unit_type). The NAL unit type is common to the codec and defines the common type of the codec. That is, the NAL unit used for the first encoding method and the NAL unit used for the second encoding method record the commonly defined NAL unit type. Fig.34In the example shown, for example, PCC DataA is coded data of codec 1, PCC DataB is coded data of codec 2, PCC MetaDataA is additional information of codec 1, and PCC MetaDataB is additional information of codec 2.

[0228] By adopting this method, the first encoding method and the second encoding method can be handled as the same codec.

[0229] Next, the encoding process of this embodiment is described. Fig.35 46 is a flowchart of the encoding process of this embodiment. The process in this figure shows the process of the first encoding unit 4630 or the second encoding unit 4650 when the above definition is used. In addition, the process in this figure is mainly composed of Figure 6 The multiplexing unit 4634 or Fig.10 The multiplexing unit 4656 shown is performed.

[0230] The process in the figure shows an example of encoding PCC data using either the second encoding method or the first encoding method, and it is known which PCC codec is used for encoding. For example, which PCC codec is used may be specified by a user or an external device.

[0231] First, the coding unit 4613 codes PCC data using any codec of the second coding method and the first coding method ( S4651 ). Next, the coding unit 4613 generates a NAL unit in a NAL unit format common to PCC ( S4652 ).

[0232] Next, the encoder 4613 sets the identifier of the PCC common NAL unit to pcc_nal_unit_type included in the NAL unit header (S4653). Next, the encoder 4613 transmits the NAL unit having the set NAL unit header and including the coded data in the payload (S4654).

[0233] Next, the decoding process of this embodiment is described. Fig.36 46 is a flowchart of the decoding process of this embodiment. The process in this figure shows the process of the first decoding unit 4640 or the second decoding unit 4660 when the above definition is used. In addition, the process in this figure is mainly composed of Figure 8 The inverse multiplexing unit 4641 or Fig.12 The inverse multiplexing unit 4661 shown is performed.

[0234] The process in the figure shows an example of encoding the PCC data using either the second encoding method or the first encoding method.

[0235] First, the decoding unit 4624 determines the codec used in encoding the data included in the NAL unit (S4661). For example, the decoding unit 4624 determines the codec used by referring to pcc_nal_unit_type included in the NAL unit header.

[0236] When the codec used is the second encoding method (the second encoding method in S4661), the decoding unit 4624 receives the NAL unit in the common format of the PCC (S4662). Next, assuming that the NAL unit is common, the decoding unit 4624 identifies the data using the common NAL unit format and the common pcc_nal_unit_type (S4663). Next, the decoding unit 4624 decodes the PCC data using the decoding process of the second encoding method (S4664).

[0237] On the other hand, when the codec used is the first encoding method (the first encoding method in S4661), the decoding unit 4624 receives the NAL unit in the common format of the PCC (S4665). Next, assuming that the NAL unit is common, the decoding unit 4624 uses the common NAL unit format and the common pcc_nal_unit_type to identify the data (S4666). Next, the decoding unit 4624 decodes the PCC data using the decoding process of the first encoding method (S4667).

[0238] Hereinafter, a description will be given of modifications of the above-mentioned Embodiments 1 to 3. As another method of indicating the PCC codec type, the following method may be used.

[0239] In Embodiment 1, Embodiment 2, and Embodiment 3, the case where two codecs, namely the first encoding method and the second encoding method, are mixed is described. However, the above-mentioned technique can also be applied when there are three or more PCC codecs.

[0240] In Embodiments 1 and 3, the identification information of the PCC codec (pcc_codec_type in Embodiment 1 and pcc_nal_unit_type in Embodiment 3b) is recorded in the NAL unit header, but the identification information of the codec may be stored in another place.

[0241] In addition, the first encoding method and the second encoding method are not limited to the above examples, and may be any codec. For example, the first encoding method and the second encoding method may be a plurality of codecs that subdivide GPCC, or may be a plurality of codecs that subdivide VPCC. For example, the first encoding method and the second encoding method may both be VPCC, and the image encoding methods used are different. The image encoding method may be, for example, AVC or HEVC. In addition, either or both of the first encoding method and the second encoding method may be encoding methods that include other encoding methods such as image, sound, and text applications.

[0242] For example, the identification information of the codec may be included in the control information included in the PCC coded stream. Here, the control information is, for example, metadata such as parameter sets or SEI (Supplemental Enhancement Information).

[0243] Fig.37 4671, and the identification information of the PCC codec is recorded in a predetermined position (e.g., parameter set) in the coded data (S4672). Next, the coding unit 4613 generates a NAL unit including the coded data, and transmits the generated NAL unit (S4673).

[0244] In addition, the identification information of the PCC codec may be defined as a profile, and the identification information of the PCC codec may be indicated in the metadata. In addition, in the case where the same codec is used for the entire sequence, the identification information of the PCC codec may also be included in the sequence parameter set. In addition, in the case where each PCC frame is encoded with a different codec, the identification information of the PCC codec may also be included in the parameter set that records the information of each frame. For example, in the case where a different codec is used for each data of the PCC, such as when the codec is different in the position information and the attribute information, the identification information of the PCC codec may also be included in the parameter set that records the information of each data. That is, the information of the codec representing the position information is included in the control information (parameter set, etc.) of the position information, and the information of the codec representing the attribute information may also be included in the control information (parameter set, etc.) of the attribute information.

[0245] In addition, the identification information of the codec may be stored in any of the above or in multiple locations. For example, the identification information of the codec may be stored in both the coded stream and the NAL unit header. In addition, when the identification information of the codec is stored in multiple locations, the same information may be stored in the multiple locations, or different information may be stored. For example, the different information is information indicating GPCC or VPCC, and information indicating any one of multiple codecs that subdivide GPCC or VPCC.

[0246] When the parameter set is included in the NAL unit, the inverse multiplexing unit 4641 or 4661 included in the decoding unit 4624 can determine whether the data included in the payload of the NAL unit is data encoded using the first encoding method or data encoded using the second encoding method by parsing the description in the parameter set. As a result, the decoding unit 4624 can filter the NAL unit that is not necessary for decoding more quickly.

[0247] Fig.38 4676). Next, the decoding unit 4624 identifies the PCC codec indicated in the specified data by parsing the specified data (e.g., the parameter set) by using the pcc_nal_unit_type included in the NAL unit header (S4677). Next, the decoding unit 4624 decodes the coded data using the identified PCC codec (S4678).

[0248] In the above, an example in which the coded stream is stored in the NAL unit is shown, but a unit (unit) of a predetermined format may be used instead of the NAL unit.

[0249] (Implementation 4)

[0250] In this embodiment, a coding unit 4670 having the functions of both the first coding unit 4630 and the second coding unit 4650 described above, and a decoding unit 4680 having the functions of both the first decoding unit 4640 and the second decoding unit 4660 are described.

[0251] Fig.39 4670 of the present embodiment. The encoding unit 4670 includes the above-mentioned first encoding unit 4630, the second encoding unit 4650, and a multiplexing unit 4671. The multiplexing unit 4671 multiplexes the encoded data generated by the first encoding unit 4630 and the encoded data generated by the second encoding unit 4650, and outputs the obtained encoded data.

[0252] Fig.40 4680 of the present embodiment. The decoding unit 4680 includes the above-mentioned first decoding unit 4640, the second decoding unit 4660, and the inverse multiplexing unit 4681. The inverse multiplexing unit 4681 extracts the coded data using the first coding method and the coded data using the second coding method from the input coded data. The inverse multiplexing unit 4681 outputs the coded data using the first coding method to the first decoding unit 4640, and outputs the coded data using the second coding method to the second decoding unit 4660.

[0253] According to the above structure, the encoding unit 4670 can selectively use the first encoding method and the second encoding method to encode the point group data. In addition, the decoding unit 4680 can decode the encoded data encoded using the first encoding method, the encoded data encoded using the second encoding method, and the encoded data encoded using both the first encoding method and the second encoding method.

[0254] For example, the coding unit 4670 may switch the coding method (the first coding method and the second coding method) in units of point cloud data or frames. In addition, the coding unit 4670 may switch the coding method in units that can be coded.

[0255] The coding unit 4670 generates coded data (coded stream) including the identification information of the PCC codec described in the first or third embodiment, for example.

[0256] The demultiplexing unit 4681 included in the decoding unit 4680 identifies the data using, for example, the identification information of the PCC codec described in Embodiment 1 or Embodiment 3. The demultiplexing unit 4681 outputs the data to the first decoding unit 4640 when the data is data encoded using the first encoding method, and outputs the data to the second decoding unit 4660 when the data is data encoded using the second encoding method.

[0257] Furthermore, the encoder 4670 may transmit, as control information, information indicating whether both encoding methods or either encoding method is used, in addition to the PCC codec identification information.

[0258] Next, the encoding process of this embodiment is described. Fig.41 : is a flowchart of the encoding process of this embodiment. By using the identification information of the PCC codec described in Embodiment 1, Embodiment 2, Embodiment 3 and the modified example, encoding processes corresponding to multiple codecs can be performed. In addition, Fig.41 Although an example in which the method of the first embodiment is used is shown, the same processing can be applied to other methods.

[0259] First, the encoding unit 4670 encodes the PCC data using a codec of one or both of the first encoding method and the second encoding method ( S4681 ).

[0260] When the codec used is the second encoding method (the second encoding method in S4682), the encoder 4670 sets the pcc_codec_type contained in the NAL unit header to a value indicating that the data contained in the payload of the NAL unit is data encoded using the second encoding method (S4683). Next, the encoder 4670 sets the identifier of the NAL unit used for the second encoding method in the pcc_nal_unit_type in the NAL unit header (S4684). Then, the encoder 4670 generates a NAL unit having the set NAL unit header and containing encoded data in the payload. Then, the encoder 4670 sends the generated NAL unit (S4685).

[0261] On the other hand, when the codec used is the first encoding method (the first encoding method in S4682), the encoder 4670 sets pcc_codec_type included in the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded using the first encoding method (S4686). Next, the encoder 4670 sets the identifier of the NAL unit used for the first encoding method to pcc_nal_unit_type included in the NAL unit header (S4687). Next, the encoder 4670 generates a NAL unit having the set NAL unit header and including encoded data in the payload. Then, the encoder 4670 sends the generated NAL unit (S4685).

[0262] Next, the decoding process of this embodiment is described. Fig.42 Flowchart of the decoding process of this embodiment. By using the identification information of the PCC codec described in Embodiment 1, Embodiment 2, Embodiment 3 and the modified example, decoding processes corresponding to multiple codecs can be performed. Fig.42 Although an example in which the method of the first embodiment is used is shown, the same processing can be applied to other methods.

[0263] First, the decoding unit 4680 receives a NAL unit (S4691). For example, this NAL unit is generated in the process in the encoding unit 4670 described above.

[0264] Next, the decoding unit 4680 determines whether pcc_codec_type included in the NAL unit header indicates the first encoding method or the second encoding method ( S4692 ).

[0265] When pcc_codec_type indicates the second coding method (the second coding method in S4692), the decoding unit 4680 determines that the data included in the payload of the NAL unit is data encoded using the second coding method (S4693). Then, the second decoding unit 4660 identifies the data using pcc_nal_unit_type included in the NAL unit header as an identifier of the NAL unit for the second coding method (S4694). Then, the decoding unit 4680 decodes the PCC data using the decoding process of the second coding method (S4695).

[0266] On the other hand, when pcc_codec_type indicates the first coding method (the first coding method in S4692), the decoding unit 4680 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4696). Then, the decoding unit 4680 identifies the data using pcc_nal_unit_type included in the NAL unit header as an identifier of the NAL unit for the first coding method (S4697). Then, the decoding unit 4680 decodes the PCC data using the decoding process of the first coding method (S4698).

[0267] As described above, a three-dimensional data encoding device according to one aspect of the present disclosure generates an encoding stream (eg, Fig.37 S4671 of the encoding stream), and storing information indicating the encoding method used for the encoding in the first encoding method and the second encoding method (for example, identification information of the codec) in the control information (for example, parameter set) of the encoding stream (for example, Fig.37 S4672).

[0268] Thus, when decoding the coded stream generated by the 3D data coding device, the 3D data decoding device can use the information stored in the control information to determine the coding method used in the coding. Therefore, even when multiple coding methods are used, the 3D data decoding device can correctly decode the coded stream.

[0269] For example, the three-dimensional data includes position information. In the encoding, the three-dimensional data encoding device encodes the position information. In the storage, the three-dimensional data encoding device stores information indicating a coding method used for coding the position information in the first coding method and the second coding method in the control information of the position information.

[0270] For example, the three-dimensional data includes position information and attribute information. In the encoding, the three-dimensional data encoding device encodes the position information and the attribute information. In the storing, the three-dimensional data encoding device stores information indicating a coding method used for coding the position information in the control information of the position information, and stores information indicating a coding method used for coding the attribute information in the control information of the attribute information.

[0271] This allows different encoding methods to be used for position information and attribute information, thereby improving encoding efficiency.

[0272] For example, the three-dimensional data encoding method further stores the encoding stream in more than one unit (for example, NAL unit) (for example, Fig.37 S4673).

[0273] For example, as in Embodiment 1 Figure 15 to Figure 18 As described in , the unit includes information, the information has a common format in the first encoding method and the second encoding method, the information is information indicating the category of data contained in the unit, and has an independent definition in the first encoding method and the second encoding method (for example, pcc_nal_unit_type).

[0274] For example, as in Embodiment 2 Figure 23 to Figure 28 As described in , the unit includes information, the information has a format independent of the first encoding method and the second encoding method, the information is information indicating the category of data included in the unit, and has a definition independent of the first encoding method and the second encoding method (for example, codec1_nal_unit_type or codec2_nal_unit_type).

[0275] For example, as in Embodiment 3 Figure 32 to Figure 34 As described in , the unit includes information, the information has a format common to the first encoding method and the second encoding method, the information is information indicating a category of data included in the unit, and has a definition common to the first encoding method and the second encoding method (for example, pcc_nal_unit_type).

[0276] For example, the three-dimensional data encoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0277] In addition, the three-dimensional data decoding device of the present embodiment determines the encoding method (for example, identification information of a codec) used in encoding the encoded stream based on information indicating the encoding method used for encoding the three-dimensional data, of the first encoding method and the second encoding method, included in control information (for example, a parameter set) of the encoded stream generated by encoding the three-dimensional data. Fig.38 S4677), decoding the coded stream using the determined coding method (for example, Fig.38 S4678).

[0278] Thus, when decoding the coded stream, the 3D data decoding device can use the information stored in the control information to determine the coding method used in the coding. Therefore, even when multiple coding methods are used, the 3D data decoding device can correctly decode the coded stream.

[0279] For example, the three-dimensional data includes position information, and the coded stream includes coded data of the position information. In the determination, the three-dimensional data decoding device determines the coding method used in the coding of the position information based on information indicating the coding method used in the coding of the position information, one of the first coding method and the second coding method, included in the control information of the position information included in the coded stream. In the decoding, the three-dimensional data decoding device decodes the coded data of the position information using the determined coding method used in the coding of the position information.

[0280] For example, the three-dimensional data includes position information and attribute information, and the coded stream includes coded data of the position information and coded data of the attribute information. In the determination, the three-dimensional data decoding device determines the coding method used in the coding of the position information based on information indicating the coding method used in the coding of the position information, which is included in the control information of the position information included in the coded stream, and determines the coding method used in the coding of the attribute information, which is included in the control information of the attribute information included in the coded stream, based on information indicating the coding method used in the coding of the attribute information, which is included in the control information of the attribute information included in the coded stream. In the decoding, the three-dimensional data decoding device decodes the coded data of the position information using the determined coding method used in the coding of the position information, and decodes the coded data of the attribute information using the determined coding method used in the coding of the attribute information.

[0281] This allows different encoding methods to be used for position information and attribute information, thereby improving encoding efficiency.

[0282] For example, the coded stream is stored in one or more units (for example, NAL units), and the three-dimensional data decoding device further obtains the coded stream from the one or more units.

[0283] For example, as in Embodiment 1 Figure 15 to Figure 18 As described in , the unit includes information, the information has a common format in the first encoding method and the second encoding method, the information is information indicating the category of data contained in the unit, and has an independent definition in the first encoding method and the second encoding method (for example, pcc_nal_unit_type).

[0284] For example, as in Embodiment 2 Figure 23 to Figure 28 As described in , the unit includes information, the information has a format independent of the first encoding method and the second encoding method, the information is information indicating the category of data included in the unit, and has a definition independent of the first encoding method and the second encoding method (for example, codec1_nal_unit_type or codec2_nal_unit_type).

[0285] For example, as in Embodiment 3 Figure 32 to Figure 34 As described in , the unit includes information, the information has a format common to the first encoding method and the second encoding method, the information is information indicating a category of data included in the unit, and has a definition common to the first encoding method and the second encoding method (for example, pcc_nal_unit_type).

[0286] For example, the three-dimensional data decoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0287] (Implementation 5)

[0288] In this embodiment, the method of storing the NAL unit in the ISOBMFF file described in Embodiment 1 is described.

[0289] ISOBMFF (ISO based media file format) is a file format standard defined by ISO / IEC 14496-12. ISOBMFF defines a format that can multiplex and store various media such as video, audio, and text, and is a standard that is independent of the media.

[0290] The basic structure (file) of ISOBMFF is explained. The basic unit in ISOBMFF is a box. A box consists of type, length, and data. The combination of boxes of various types is called a file.

[0291] Fig.43 This is a diagram showing the basic structure (file) of ISOBMFF. The ISOBMFF file mainly includes boxes such as ftyp which indicates the file version (brand) using 4CC (4 character code), moov which stores metadata such as control information, and mdat which stores data.

[0292] The method of saving each media in the ISOBMFF file is separately specified. For example, the method of saving AVC video and HEVC video is specified by ISO / IEC14496-15. Here, in order to store or transmit PCC coded data, it is conceivable to use the ISOBMFF function extension, but there is no provision for saving PCC coded data in the ISOBMFF file. Therefore, in this embodiment, the method of saving PCC coded data in the ISOBMFF file is described.

[0293] Fig.44 This is a diagram showing a protocol stack when a NAL unit common to a PCC codec is saved in an ISOBMFF file. Here, a NAL unit common to a PCC codec is saved in an ISOBMFF file. NAL units are common to PCC codecs, but since multiple PCC codecs are saved in a NAL unit, it is desirable to define a saving method corresponding to each codec (Carriage of Codec1, Carriage of Codec2).

[0294] Next, a method of storing a common PCC NAL unit supporting a plurality of PCC codecs in an ISOBMFF file is described. Fig.45 This diagram shows an example of storing a common PCC NAL unit in a file of ISOBMFF of the storage method (Carriage of Codec1) of codec 1. Fig.46 This diagram shows an example of storing a common PCC NAL unit in an ISOBMFF file of the storage method (Carriage of Codec2) of codec 2.

[0295] Here, ftyp is important information for identifying the file format, and a different identifier for each codec is defined as ftyp. When PCC coded data encoded by the first coding method (coding mode) is saved in a file, ftyp=pcc1 is set. When PCC coded data encoded by the second coding method is saved in a file, ftyp=pcc2 is set.

[0296] Here, pcc1 indicates codec 1 using PCC (first encoding method). pcc2 indicates codec 2 using PCC (second encoding method). That is, pcc1 and pcc2 indicate that the data is PCC (coded data of three-dimensional data (point cloud data)), and indicate PCC codec (first encoding method and second encoding method).

[0297] The following describes a method of storing the NAL unit in an ISOBMFF file. The multiplexing unit parses the NAL unit header, and when pcc_codec_type=Codec1, pcc1 is written in the ISOBMFF ftyp.

[0298] Furthermore, the multiplexing unit parses the NAL unit header, and when pcc_codec_type=Codec2, pcc2 is recorded in ftyp of ISOBMFF.

[0299] Furthermore, when pcc_nal_unit_type is metadata, the multiplexing unit stores the NAL unit in, for example, moov or mdat using a predetermined method. When pcc_nal_unit_type is data, the multiplexing unit stores the NAL unit in, for example, moov or mdat using a predetermined method.

[0300] For example, the multiplexing unit may store the NAL unit size in the NAL unit similarly to HEVC.

[0301] By parsing the ftyp contained in the file in the inverse multiplexing unit (system layer) using this storage method, it is possible to determine whether the PCC coded data is encoded using the first coding method or the second coding method. Furthermore, as described above, by determining whether the PCC coded data is encoded using the first coding method or the second coding method, it is possible to extract coded data encoded using one of the coding methods from data in which coded data encoded using both coding methods are mixed. Thus, when transmitting coded data, the amount of data transmitted can be suppressed. In addition, by using this storage method, it is possible to use a common data format instead of setting different data (file) formats in the first coding method and the second coding method.

[0302] In addition, when the identification information of the codec is indicated in metadata of the system layer such as ftyp in ISOBMFF, the multiplexing unit may store the NAL unit after deleting pcc_nal_unit_type in the ISOBMFF file.

[0303] Next, the structure and operation of the multiplexing unit of the three-dimensional data encoding system (three-dimensional data encoding device) according to the present embodiment and the demultiplexing unit of the three-dimensional data decoding system (three-dimensional data decoding device) according to the present embodiment are described.

[0304] Fig.47 47 is a diagram showing the structure of the first multiplexing unit 4710. The first multiplexing unit 4710 includes a file conversion unit 4711 that generates multiplexed data (file) by storing the coded data and control information (NAL unit) generated by the first coding unit 4630 in an ISOBMFF file. The first multiplexing unit 4710 includes, for example, Figure 1 In the multiplexing unit 4614 shown.

[0305] Fig.48 The first demultiplexing unit 4720 includes a file inverse conversion unit 4721 that obtains coded data and control information (NAL unit) from multiplexed data (file) and outputs the obtained coded data and control information to the first decoding unit 4640. The first demultiplexing unit 4720 includes, for example, Figure 1 In the inverse multiplexing unit 4623 shown.

[0306] Fig.49 4730 is a diagram showing the structure of the second multiplexing unit 4730. The second multiplexing unit 4730 includes a file conversion unit 4731 that generates multiplexed data (file) by storing the coded data and control information (NAL unit) generated by the second coding unit 4650 in an ISOBMFF file. The second multiplexing unit 4730 is included in, for example, Figure 1 In the multiplexing unit 4614 shown.

[0307] Fig.50 The second demultiplexing unit 4740 is a diagram showing the structure of the second demultiplexing unit 4740. The second demultiplexing unit 4740 includes a file inverse conversion unit 4741 that obtains coded data and control information (NAL unit) from the multiplexed data (file) and outputs the obtained coded data and control information to the second decoding unit 4660. The second demultiplexing unit 4740 includes, for example, Figure 1 In the inverse multiplexing unit 4623 shown.

[0308] Fig.51This is a flowchart of the multiplexing process performed by the first multiplexing unit 4710. First, the first multiplexing unit 4710 analyzes pcc_codec_type included in the NAL unit header to determine whether the codec used is the first encoding method or the second encoding method (S4701).

[0309] When pcc_codec_type indicates the second encoding method (the second encoding method in S4702), the first multiplexing unit 4710 does not process the NAL unit (S4703).

[0310] On the other hand, when pcc_codec_type indicates the second coding method (the first coding method in S4702), the first multiplexer 4710 records pcc1 in ftyp (S4704). That is, the first multiplexer 4710 records information indicating that data encoded by the first coding method is stored in the file in ftyp.

[0311] Next, the first multiplexing unit 4710 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_nal_unit_type (S4705). In addition, the first multiplexing unit 4710 creates an ISOBMFF file including the ftyp and the box (S4706).

[0312] Fig.52 This is a flowchart of the multiplexing process performed by the second multiplexing unit 4730. First, the second multiplexing unit 4730 analyzes pcc_codec_type included in the NAL unit header to determine whether the codec used is the first encoding method or the second encoding method (S4711).

[0313] When pcc_unit_type indicates the second coding method (the second coding method in S4712), the second multiplexing unit 4730 records pcc2 in ftyp (S4713). That is, the second multiplexing unit 4730 records information indicating that data encoded by the second coding method is stored in the file in ftyp.

[0314] Next, the second multiplexing unit 4730 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_nal_unit_type (S4714). Furthermore, the second multiplexing unit 4730 creates an ISOBMFF file including the ftyp and the box (S4715).

[0315] On the other hand, when pcc_unit_type indicates the first encoding method (the first encoding method in S4712), the second multiplexing unit 4730 does not process the NAL unit (S4716).

[0316] In addition, the above-mentioned process shows an example of encoding PCC data using either the first encoding method or the second encoding method. The first multiplexing unit 4710 and the second multiplexing unit 4730 save the desired NAL unit to the file by identifying the codec type of the NAL unit. In addition, in the case where the identification information of the PCC codec is included in addition to the NAL unit header, the first multiplexing unit 4710 and the second multiplexing unit 4730 may also use the identification information of the PCC codec included in addition to the NAL unit header to identify the codec type (the first encoding method or the second encoding method) in steps S4701 and S4711.

[0317] Furthermore, when the first multiplexing unit 4710 and the second multiplexing unit 4730 save the data in the file in steps S4706 and S4714, they may delete pcc_nal_unit_type from the NAL unit header and save the data in the file.

[0318] Fig.53 This is a flowchart showing the processing performed by the first demultiplexing unit 4720 and the first decoding unit 4640. First, the first demultiplexing unit 4720 parses the ftyp contained in the ISOBMFF file (S4721). In the case where the codec represented by ftyp is the second coding method (pcc2) (it is the second coding method in S4722), the first demultiplexing unit 4720 determines that the data contained in the payload of the NAL unit is data encoded using the second coding method (S4723). In addition, the first demultiplexing unit 4720 passes the determination result to the first decoding unit 4640. The first decoding unit 4640 does not process the NAL unit (S4724).

[0319] On the other hand, when the codec indicated by ftyp is the first coding method (pcc1) (the first coding method in S4722), the first demultiplexing unit 4720 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4725). In addition, the first demultiplexing unit 4720 transmits the determination result to the first decoding unit 4640.

[0320] The first decoding unit 4640 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first encoding method (S4726). Then, the first decoding unit 4640 decodes the PCC data using the decoding process of the first encoding method (S4727).

[0321] Fig.54 4731. This is a flowchart showing the processing performed by the second demultiplexing unit 4740 and the second decoding unit 4660. First, the second demultiplexing unit 4740 parses the ftyp contained in the ISOBMFF file (S4731). When the codec represented by ftyp is the second coding method (pcc2) (the second coding method in S4732), the second demultiplexing unit 4740 determines that the data contained in the payload of the NAL unit is data encoded using the second coding method (S4733). In addition, the second demultiplexing unit 4740 passes the determination result to the second decoding unit 4660.

[0322] The second decoding unit 4660 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the second encoding method (S4734). Then, the second decoding unit 4660 decodes the PCC data using the decoding process of the second encoding method (S4735).

[0323] On the other hand, when the codec indicated by ftyp is the first coding method (pcc1) (the first coding method in S4732), the second demultiplexing unit 4740 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4736). In addition, the second demultiplexing unit 4740 passes the determination result to the second decoding unit 4660. The second decoding unit 4660 does not process the NAL unit (S4737).

[0324] In this way, for example, by identifying the codec type of the NAL unit in the first demultiplexing unit 4720 or the second demultiplexing unit 4740, the codec type can be identified at an earlier stage. Furthermore, the desired NAL unit can be input to the first decoding unit 4640 or the second decoding unit 4660, and unnecessary NAL units can be removed. In this case, in the first decoding unit 4640 or the second decoding unit 4660, it may be unnecessary to parse the identification information of the codec. In addition, it is also possible to parse the identification information of the codec again by referring to the NAL unit type in the first decoding unit 4640 or the second decoding unit 4660.

[0325] Furthermore, when pcc_nal_unit_type is deleted from the NAL unit header in the first multiplexing unit 4710 or the second multiplexing unit 4730 , the first inverse multiplexing unit 4720 or the second inverse multiplexing unit 4740 may output the NAL unit to the first decoding unit 4640 or the second decoding unit 4660 after assigning pcc_nal_unit_type to the NAL unit.

[0326] (Implementation 6)

[0327] In this embodiment, a multiplexing unit and a demultiplexing unit corresponding to the encoding unit 4670 and the decoding unit 4680 corresponding to a plurality of codecs described in Embodiment 4 are described. Fig.55 This is a diagram showing the configuration of the encoding unit 4670 and the third multiplexing unit 4750 according to this embodiment.

[0328] The coding unit 4670 codes the point cloud data using one or both of the first coding method and the second coding method. The coding unit 4670 may switch the coding method (the first coding method and the second coding method) in units of point cloud data or frames. In addition, the coding unit 4670 may switch the coding method in units that can be coded.

[0329] The coding unit 4670 generates coded data (coded stream) including the identification information of the PCC codec described in Embodiments 1 to 4.

[0330] The third multiplexing unit 4750 includes a file conversion unit 4751. The file conversion unit 4751 converts the NAL unit output from the encoding unit 4670 into a file of PCC data. The file conversion unit 4751 parses the codec identification information included in the NAL unit header to determine whether the PCC coded data is data encoded by the first coding method, data encoded by the second coding method, or data encoded by both methods. The file conversion unit 4751 records the version name that can identify the codec in ftyp. For example, in the case of indicating encoding by both methods, pcc3 is recorded in ftyp.

[0331] Furthermore, when the encoding unit 4670 describes the identification information of the PCC codec other than the NAL unit, the file conversion unit 4751 may determine the PCC codec (encoding method) using the identification information.

[0332] Fig.56 This is a diagram showing the configuration of the third inverse multiplexing unit 4760 and the decoding unit 4680 according to the present embodiment.

[0333] The third demultiplexing unit 4760 includes a file inverse conversion unit 4761. The file inverse conversion unit 4761 analyzes ftyp included in the file and determines whether the PCC coded data is data coded using the first coding method, data coded using the second coding method, or data coded using both methods.

[0334] When the PCC coded data is coded using one of the coding methods, the data is input to the corresponding decoding unit of the first decoding unit 4640 and the second decoding unit 4660, and no data is input to the other decoding unit. When the PCC coded data is coded using both coding methods, the data is input to the decoding unit 4680 corresponding to both methods.

[0335] The decoding unit 4680 decodes the PCC coded data using one or both of the first coding method and the second coding method.

[0336] Fig.57 It is a flowchart showing the processing performed by the third multiplexing unit 4750 related to this embodiment.

[0337] First, the third multiplexing unit 4750 analyzes pcc_codec_type included in the NAL unit header to determine whether the codec used is the first encoding method, the second encoding method, or both the first encoding method and the second encoding method (S4741).

[0338] When the second encoding method is used (Yes in S4742, and the second encoding method is used in S4743), the third multiplexing unit 4750 records pcc2 in ftyp (S4744). That is, the third multiplexing unit 4750 records information indicating that data encoded by the second encoding method is stored in the file in ftyp.

[0339] Next, the third multiplexing unit 4750 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_unit_type (S4745). Furthermore, the third multiplexing unit 4750 creates an ISOBMFF file including the ftyp and the box (S4746).

[0340] On the other hand, when the first encoding method is used ("Yes" in S4742, and the first encoding method is used in S4743), the third multiplexing unit 4750 records pcc1 in ftyp (S4747). That is, the third multiplexing unit 4750 records information indicating that data encoded by the first encoding method is stored in the file in ftyp.

[0341] Next, the third multiplexing unit 4750 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_unit_type (S4748). Furthermore, the third multiplexing unit 4750 creates an ISOBMFF file including the ftyp and the box (S4746).

[0342] On the other hand, when both the first encoding method and the second encoding method are used (No in S4742), the third multiplexing unit 4750 records pcc3 in ftyp (S4749). That is, the third multiplexing unit 4750 records information indicating that data encoded by both encoding methods is stored in the file in ftyp.

[0343] Next, the third multiplexing unit 4750 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_unit_type (S4750). Furthermore, the third multiplexing unit 4750 creates an ISOBMFF file including the ftyp and the box (S4746).

[0344] Fig.58 4760 and the decoding unit 4680. First, the third demultiplexing unit 4760 parses the ftyp contained in the ISOBMFF file (S4761). In the case where the codec represented by ftyp is the second coding method (pcc2) ("yes" in S4762, and it is the second coding method in S4763), the third demultiplexing unit 4760 determines that the data contained in the payload of the NAL unit is data encoded using the second coding method (S4764). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0345] The decoding unit 4680 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the second encoding method (S4765). Then, the decoding unit 4680 decodes the PCC data using the decoding process of the second encoding method (S4766).

[0346] On the other hand, when the codec indicated by ftyp is the first coding method (pcc1) ("Yes" in S4762, and the first coding method in S4763), the third demultiplexing unit 4760 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4767). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0347] The decoding unit 4680 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first encoding method (S4768). Then, the decoding unit 4680 decodes the PCC data using the decoding process of the first encoding method (S4769).

[0348] On the other hand, when ftyp indicates that both coding methods (pcc3) are used (No in S4762), the third demultiplexing unit 4760 determines that the data included in the payload of the NAL unit is data encoded using both the first coding method and the second coding method (S4770). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0349] The decoding unit 4680 identifies the data by setting the pcc_nal_unit_type included in the NAL unit header to be the identifier of the NAL unit for the codec described in pcc_codec_type (S4771). Furthermore, the decoding unit 4680 decodes the PCC data using the decoding processing of both encoding methods (S4772). That is, the decoding unit 4680 decodes the data encoded by the first encoding method using the decoding processing of the first encoding method, and decodes the data encoded by the second encoding method using the decoding processing of the second encoding method.

[0350] The following describes a modified example of the present embodiment. As the type of version represented by ftyp, the following types may be represented by identification information. In addition, a combination of the following types may be represented by identification information.

[0351] The identification information indicates whether the object of the original data before PCC encoding is a point group with a limited area or a large-scale point group with an unlimited area like map information.

[0352] The identification information may also indicate whether the original data before PCC encoding is a static object or a dynamic object.

[0353] As described above, the identification information may indicate whether the PCC coded data is data coded using the first coding method or data coded using the second coding method.

[0354] The identification information may indicate an algorithm used in PCC encoding. Here, the algorithm is, for example, an encoding method that can be used in the first encoding method or the second encoding method.

[0355] The identification information may also indicate the difference in the method of storing the PCC coded data in the ISOBMFF file. For example, the identification information may indicate whether the storage method used is a storage method for accumulation or a storage method for real-time transmission such as dynamic streaming.

[0356] In addition, in Implementation 5, the method for saving the NAL unit described in Implementation 1 is described, and in Implementation 6, the method for saving the NAL unit described in Implementation 4 is described, but the same method for saving the NAL unit described in Implementation 2 and Implementation 3 can also be applied to the same method for saving the NAL unit, thereby saving the identification information of the PCC codec in the ISOBMFF file.

[0357] In addition, in Embodiments 5 and 6, an example of using ISOBMFF as a file format is described, but other methods may be used. For example, the same method as in this embodiment may be used when saving PCC encoded data to MPEG-2TS Systems, MPEG-DASH, MMT, or RMP.

[0358] In the above, an example is shown in which metadata such as identification information is stored in ftyp, but these metadata may be stored outside of ftyp. For example, these metadata may be stored in moov.

[0359] As described above, the three-dimensional data storage device (or three-dimensional data multiplexing device, or three-dimensional data encoding device) performs Fig.59 Processing shown.

[0360] First, the three-dimensional data storage device (for example, including the first multiplexing unit 4710, the second multiplexing unit 4730 or the third multiplexing unit 4750) obtains one or more units (for example, NAL units) storing a coded stream obtained by encoding point group data (S4781). Next, the three-dimensional data storage device saves the one or more units to a file (for example, an ISOBMFF file) (S4782). In addition, during the saving (S4782), the three-dimensional data storage device saves information indicating that the data stored in the file is data obtained by encoding point group data (for example, pcc1, pcc2 or pcc3) in the control information (for example, ftyp) of the above file.

[0361] Thus, in a device that processes a file generated by the three-dimensional data storage device, it is possible to refer to the control information of the file and determine at an early stage whether the data stored in the file is coded data of point group data, thereby reducing the processing amount of the device or speeding up the processing.

[0362] For example, the information also indicates the encoding method used in encoding the point group data in the first encoding method and the second encoding method. In addition, the data stored in the file is the data obtained by encoding the point group data, and the encoding method used in encoding the point group data in the first encoding method and the second encoding method, which can be represented by a single information or different information.

[0363] Thus, in a device that processes a file generated by the three-dimensional data storage device, the codec used for the data stored in the file can be determined earlier by referring to the file control information, thereby reducing the amount of processing of the device or speeding up the processing.

[0364] For example, the first encoding method is a method (GPCC) of encoding position information of the point group data represented by an N (N is an integer greater than 2) fork tree and using the position information to encode attribute information, and the second encoding method is a method (VPCC) of generating a two-dimensional image based on the point group data and encoding the two-dimensional image using an image encoding method.

[0365] For example, the above file is based on ISOBMFF (ISO based media file format: ISO base media file format).

[0366] For example, the three-dimensional data storage device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0367] In addition, as described above, the three-dimensional data obtaining device (or three-dimensional data demultiplexing device, or three-dimensional data decoding device) performs Fig.60 Processing shown.

[0368] The three-dimensional data obtaining device (for example, including the first inverse multiplexing unit 4720, the second inverse multiplexing unit 4740 or the third inverse multiplexing unit 4760) obtains a file (for example, an ISOBMFF file) storing one or more units (for example, NAL units), wherein the one or more units store a coded stream obtained by encoding point group data (S4791). Next, the three-dimensional data obtaining device obtains one or more units from the file (S4792). In addition, the control information of the file (for example, ftyp) includes information indicating that the data stored in the file is data obtained by encoding the point group data (for example, pcc1, pcc2 or pcc3).

[0369] For example, the three-dimensional data obtaining device refers to the above information to determine whether the data stored in the file is data obtained by encoding the point group data. In addition, when the three-dimensional data obtaining device determines that the data stored in the file is data obtained by encoding the point group data, it generates the point group data by decoding the data obtained by encoding the point group data contained in one or more units. Alternatively, when the three-dimensional data obtaining device determines that the data stored in the file is data obtained by encoding the point group data, it outputs (notifies) information indicating that the data contained in one or more units is data obtained by encoding the point group data to a subsequent processing unit (for example, the first decoding unit 4640, the second decoding unit 4660, or the decoding unit 4680).

[0370] Thus, the 3D data acquisition device can refer to the control information of the file and determine whether the data stored in the file is the coded data of the point group data at an early stage, thereby reducing the processing amount of the 3D data acquisition device or the subsequent device or speeding up the processing.

[0371] For example, the information also indicates the encoding method used in the encoding of the first encoding method and the second encoding method. In addition, the data stored in the file is the data obtained by encoding the point group data, and the encoding method used in the encoding of the point group data of the first encoding method and the second encoding method, which can be represented by a single information or different information.

[0372] Thus, the 3D data acquisition device can refer to the control information of the file and determine the codec used for the data stored in the file at an early stage, thereby reducing the processing amount or speeding up the processing of the 3D data acquisition device or the subsequent device.

[0373] For example, the three-dimensional data obtaining device obtains data encoded by one of the encoding methods from the encoded point cloud data including data encoded by the first encoding method and data encoded by the second encoding method based on the above information.

[0374] For example, the first encoding method is a method (GPCC) of encoding position information of the point group data represented by an N (N is an integer greater than 2) fork tree and using the position information to encode attribute information, and the second encoding method is a method (VPCC) of generating a two-dimensional image based on the point group data and encoding the two-dimensional image using an image coding method.

[0375] For example, the above file is based on ISOBMFF (ISO based media file format).

[0376] For example, the three-dimensional data acquisition device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0377] As mentioned above, although the three-dimensional data storage device and the three-dimensional data obtaining device etc. which are embodiment of this disclosure are demonstrated, this disclosure is not limited to this embodiment.

[0378] In addition, each processing unit included in the three-dimensional data storage device and the three-dimensional data acquisition device of the above-mentioned embodiment can be typically implemented as an LSI of an integrated circuit. These can be made into one chip separately, or part or all of them can be made into one chip.

[0379] Furthermore, integrated circuits are not limited to LSIs, and can be implemented by dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays) that are programmable after LSI manufacturing, or reconfigurable processors that can reconfigure the connections or settings of circuits within LSIs can also be used.

[0380] Furthermore, in each of the above-mentioned embodiments, each component may be formed by dedicated hardware, or may be implemented by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0381] Furthermore, the present disclosure can be implemented as a three-dimensional data storage method or a three-dimensional data acquisition method, etc., which are executed by a three-dimensional data storage device or a three-dimensional data acquisition device, etc.

[0382] Furthermore, the division of the functional blocks in the block diagram is an example, and multiple functional blocks can be implemented as one functional block, and one functional block can also be divided into multiple blocks, and a part of the functions can also be moved to other functional blocks. Furthermore, the functions of multiple functional blocks with similar functions can also be processed in parallel or in time division by a single hardware or software.

[0383] Furthermore, the execution order of each step in the flowchart is an example given for the purpose of specifically describing the present disclosure, and may be an order other than the above. Furthermore, part of the above steps may be executed simultaneously (in parallel) with other steps.

[0384] The above describes one or more forms of a three-dimensional data storage device and a three-dimensional data acquisition device based on the embodiments, but the present disclosure is not limited to these embodiments. Within the scope of the present disclosure, various forms obtained by performing various modifications that can be thought of by those skilled in the art on the present embodiment and forms obtained by combining constituent elements in different embodiments are all included in the scope of one or more forms.

[0385] Industrial Applicability

[0386] The present disclosure is applicable to a three-dimensional data storage device and a three-dimensional data acquisition device.

[0387] Description of the attached figure

[0388] 4601 Three-dimensional data encoding system

[0389] 4602 3D data decoding system

[0390] 4603 Sensor Terminal

[0391] 4604 External connection

[0392] 4611 Point Group Data Generation System

[0393] 4612 Prompt Department

[0394] 4613 Coding Department

[0395] 4614 Multiplexing Department

[0396] 4615 Input and Output

[0397] 4616 Control Department

[0398] 4617 Sensor Information Acquisition Department

[0399] 4618 Point Group Data Generation Department

[0400] 4621 Sensor Information Acquisition Department

[0401] 4622 Input and Output

[0402] 4623 Inverse Multiplexing Department

[0403] 4624 Decoding Department

[0404] 4625 Prompt Department

[0405] 4626 User Interface

[0406] 4627 Control Department

[0407] 4630 1st Coding Department

[0408] 4631 Position Information Coding Unit

[0409] 4632 Attribute Information Coding Unit

[0410] 4633 Additional Information Coding Department

[0411] 4634 Multiplexing Department

[0412] 4640 Decoding Unit 1

[0413] 4641 Inverse Multiplexing Department

[0414] 4642 Position information decoding unit

[0415] 4643 Attribute information decoding unit

[0416] 4644 Additional information decoding unit

[0417] 4650 No. 2 Coding Department

[0418] 4651 Additional Information Generation Department

[0419] 4652 Position Image Generation Unit

[0420] 4653 Attribute Image Generation Unit

[0421] 4654 Video Coding Department

[0422] 4655 Additional Information Coding Department

[0423] 4656 Multiplexing Department

[0424] 4660 Decoding Unit 2

[0425] 4661 Inverse Multiplexing Department

[0426] 4662 Image Decoding Department

[0427] 4663 Additional information decoding unit

[0428] 4664 Position Information Generation Unit

[0429] 4665 Attribute Information Generation Unit

[0430] 4670 Coding Department

[0431] 4671 Multiplexing Department

[0432] 4680 Decoding Department

[0433] 4681 Inverse Multiplexing Department

[0434] 4710 No.1 Multiplexing Department

[0435] 4711 File Conversion Department

[0436] 4720 1st Inverse Multiplexing Unit

[0437] 4721 File Inverse Conversion Unit

[0438] 4730 Second Multiplexing Unit

[0439] 4731 File Conversion Department

[0440] 4740 Second inverse multiplexing unit

[0441] 4741 File Inverse Conversion Unit

[0442] 4750 No. 3 Multiplexing Unit

[0443] 4751 File Conversion Department

[0444] 4760 The 3rd Inverse Multiplexing Unit

[0445] 4761 File Inverse Transformation Unit

Claims

1. A method for preserving three-dimensional data, wherein: obtaining one or more units storing encoding information after encoding three-dimensional data, The one or more units are saved in a file. The file indicates a coding method used for coding the three-dimensional data from among a plurality of coding methods including a first coding method and a second coding method. When the three-dimensional data is encoded by the first encoding method, the generated encoding information is stored in a first unit having a first format. When the three-dimensional data is encoded by the second encoding method, the generated encoding information is stored in a second unit having a second format, the second format being different from the first format. The first unit and the second unit respectively include a header and a payload, the payload includes encoded data or metadata, and the header includes information indicating a data type of the payload.

2. The three-dimensional data storage method according to claim 1, wherein: The file includes information indicating that the data stored in the file is data generated by encoding the three-dimensional data.

3. The three-dimensional data storage method according to claim 1, wherein: The first encoding method is a method of encoding position information of the position of the three-dimensional data represented by an N-ary tree, and encoding the attribute information using the position information, wherein N is an integer greater than 2. The second encoding method is a method of generating a two-dimensional image from three-dimensional data and encoding the two-dimensional image using a video encoding method.

4. The three-dimensional data storage method according to any one of claims 1 to 3, wherein: The file complies with ISOBMFF, the ISO Base Media File Format.

5. The three-dimensional data storage method according to claim 4, wherein: The three-dimensional data includes a point group.

6. A method for obtaining three-dimensional data, wherein: A file storing one or more units is obtained, wherein the one or more units store encoding information after encoding three-dimensional data. obtaining said one or more units from said file, The file indicates a coding method used for coding the three-dimensional data from among a plurality of coding methods including a first coding method and a second coding method. When the three-dimensional data is encoded by the first encoding method, the generated encoding information is stored in a first unit having a first format. When the three-dimensional data is encoded by the second encoding method, the generated encoding information is stored in a second unit having a second format, the second format being different from the first format. The first unit and the second unit respectively include a header and a payload, the payload includes encoded data or metadata, and the header includes information indicating a data type of the payload.

7. The three-dimensional data acquisition method according to claim 6, wherein: The file includes information indicating that the data stored in the file is data generated by encoding the three-dimensional data.

8. The three-dimensional data acquisition method according to claim 6, wherein: The first encoding method is a method of encoding position information of the position of the three-dimensional data represented by an N-ary tree, and encoding the attribute information using the position information, wherein N is an integer greater than 2. The second encoding method is a method of generating a two-dimensional image from three-dimensional data and encoding the two-dimensional image using a video encoding method.

9. The three-dimensional data acquisition method according to any one of claims 6 to 8, wherein: The file complies with ISOBMFF, the ISO Base Media File Format.

10. The three-dimensional data acquisition method according to claim 9, wherein: The three-dimensional data includes a point group.

11. A three-dimensional data storage device, wherein: have: Processor; and Memory, The processor uses the memory, obtaining one or more units storing encoding information after encoding three-dimensional data, The one or more units are saved in a file. The file indicates a coding method used for coding the three-dimensional data from among a plurality of coding methods including a first coding method and a second coding method. When the three-dimensional data is encoded by the first encoding method, the generated encoding information is stored in a first unit having a first format. When the three-dimensional data is encoded by the second encoding method, the generated encoding information is stored in a second unit having a second format, the second format being different from the first format. The first unit and the second unit respectively include a header and a payload, the payload includes encoded data or metadata, and the header includes information indicating a data type of the payload.

12. A three-dimensional data acquisition device, wherein: have: Processor; and Memory, The processor uses the memory, A file storing one or more units is obtained, wherein the one or more units store encoding information after encoding three-dimensional data. obtaining said one or more units from said file, The file indicates a coding method used for coding the three-dimensional data from among a plurality of coding methods including a first coding method and a second coding method. When the three-dimensional data is encoded by the first encoding method, the generated encoding information is stored in a first unit having a first format. When the three-dimensional data is encoded by the second encoding method, the generated encoding information is stored in a second unit having a second format, the second format being different from the first format. The first unit and the second unit respectively include a header and a payload, the payload includes encoded data or metadata, and the header includes information indicating a data type of the payload.

Images