Three-dimensional data encoding method, decoding method, encoding device and decoding device

Abstract

The invention provides a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device and a three-dimensional data decoding device. A three-dimensional data encoding method generates a plurality of encoded data by encoding a plurality of subspaces obtained by dividing an object space including a plurality of three-dimensional points, and generates a bitstream including first information indicating whether or not the plurality of subspaces overlap.

Description

[0001] This application is a division of an invention patent application with an application date of August 29, 2019, application number 201980055810.8, and invention name “Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device and three-dimensional data decoding device”. Technical Field

[0002] The present invention relates to a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device and a three-dimensional data decoding device. Background Art

[0003] In the future, devices and services that make use of 3D data will become more common in large fields such as computer vision, map information, monitoring, infrastructure inspection, or image distribution, which are used for autonomous operation of cars or robots. 3D data is obtained by various methods such as distance sensors such as rangefinders, stereo cameras, or a combination of multiple single-lens reflex cameras.

[0004] As one of the methods of expressing three-dimensional data, there is a method called point cloud, which expresses the shape of a three-dimensional structure through a group of points in a three-dimensional space. The position and color of the point group are saved in the point cloud. Although point cloud is expected to become the mainstream method of expressing three-dimensional data, the amount of point group data is very large. Therefore, in the accumulation or transmission of three-dimensional data, it is necessary to compress the data volume through encoding, just like two-dimensional moving images (for example, MPEG-4AVC or HEVC standardized by MPEG).

[0005] In addition, compression of point clouds is partially supported by a public library (PointCloud Library) that performs point cloud association processing.

[0006] Furthermore, there is known a technique for searching for facilities located around a vehicle using three-dimensional map data and displaying the facilities (for example, see Patent Document 1).

[0007] Prior art literature

[0008] Patent Literature

[0009] Patent Document 1: International Publication No. 2014 / 020663 Summary of the invention

[0010] Problems to be solved by the invention

[0011] In the encoding and decoding of three-dimensional data, it is desired to improve the encoding efficiency.

[0012] An object of the present invention is to provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device or a three-dimensional data decoding device capable of improving encoding efficiency.

[0013] Means used to solve problems

[0014] A three-dimensional data encoding method according to a technical solution of the present invention generates a plurality of encoded data by encoding a plurality of subspaces obtained by dividing an object space including a plurality of three-dimensional points, and generates a bit stream including first information, wherein the first information indicates whether the plurality of subspaces are repeated.

[0015] A three-dimensional data decoding method according to a technical solution of the present invention restores the multiple subspaces by decoding multiple encoded data contained in a bit stream, which is generated by encoding multiple subspaces formed by dividing an object space including multiple three-dimensional points. The bit stream includes first information indicating whether the multiple subspaces are repeated.

[0016] Effects of the Invention

[0017] The present invention can provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device or a three-dimensional data decoding device that can improve encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a diagram showing the structure of a three-dimensional data encoding and decoding system according to the first embodiment.

[0019] Figure 2 This is a diagram showing an example of the structure of point cloud data related to Implementation Example 1.

[0020] Figure 3 This is a diagram showing an example of the structure of a data file recording point group data information related to Implementation Example 1.

[0021] Figure 4 This is a diagram showing types of point cloud data related to Implementation Example 1.

[0022] Figure 5 This is a diagram showing the structure of the first encoding unit according to the first embodiment.

[0023] Figure 6 This is a block diagram of the first encoding unit related to implementation mode 1.

[0024] Figure 7 This is a diagram showing the structure of the first decoding unit according to the first embodiment.

[0025] Figure 8 This is a block diagram of the first decoding unit related to implementation mode 1.

[0026] Fig. 9 This is a diagram showing the structure of the second encoding unit related to Implementation Example 1.

[0027] Fig.10 This is a block diagram of the second encoding unit related to implementation mode 1.

[0028] Fig.11 This is a diagram showing the structure of the second decoding unit related to Implementation Example 1.

[0029] Fig.12 This is a block diagram of the second decoding unit related to implementation mode 1.

[0030] Fig.13 This is a diagram showing a protocol stack regarding PCC coded data according to the first embodiment.

[0031] Fig.14 This is a block diagram of the encoding unit related to implementation mode 1.

[0032] Fig.15 This is a block diagram of a decoding unit related to implementation mode 1.

[0033] Fig.16 This is a flowchart of the encoding process related to implementation mode 1.

[0034] Fig.17 This is a flowchart of the decoding process related to implementation mode 1.

[0035] Fig.18 This is a diagram showing the basic structure of ISOBMFF related to implementation example 2.

[0036] Fig.19 This is a diagram showing a protocol stack related to Implementation Example 2.

[0037] Fig. 20 This is a diagram showing an example of storing NAL units in a file for codec 1 according to the second embodiment.

[0038] Fig.21 This is a diagram showing an example of storing the NAL unit related to Implementation Example 2 in a file for codec 2.

[0039] Fig. 22 This is a diagram showing the structure of the first multiplexing unit related to Implementation Example 2.

[0040] Fig.23 This is a diagram showing the structure of the first inverse multiplexing unit related to Implementation Example 2.

[0041] Fig.24 This is a diagram showing the structure of the second multiplexing unit related to Implementation Example 2.

[0042] Fig.25This is a diagram showing the structure of the second inverse multiplexing unit related to Implementation Example 2.

[0043] Fig.26 This is a flowchart of the processing performed by the first multiplexing unit in the second implementation mode.

[0044] Fig. 27 This is a flowchart of the processing performed by the second multiplexing unit in implementation mode 2.

[0045] Fig.28 This is a flowchart of the processing performed by the first demultiplexing unit and the first decoding unit in accordance with the second embodiment.

[0046] Fig.29 This is a flowchart showing the processing performed by the second demultiplexing unit and the second decoding unit related to implementation mode 2.

[0047] Fig.30 This is a diagram showing the structure of the encoding unit and the third multiplexing unit related to the third implementation mode.

[0048] Fig.31 This is a diagram showing the structure of the third inverse multiplexing unit and decoding unit related to implementation mode 3.

[0049] Fig.32 This is a flowchart of the processing performed by the third multiplexing unit in implementation mode 3.

[0050] Fig.33 This is a flowchart of the processing performed by the third demultiplexing unit and decoding unit in accordance with the third implementation mode.

[0051] Fig.34 This is a flowchart of the processing performed by the three-dimensional data storage device according to the third embodiment.

[0052] Fig.35 This is a flowchart of the processing performed by the three-dimensional data acquisition device according to the third embodiment.

[0053] Fig.36 This is a diagram showing the structure of the encoding unit and the multiplexing unit related to Implementation Example 4.

[0054] Fig.37 This is a diagram showing an example of the structure of encoded data related to Implementation Example 4.

[0055] Fig.38 This is a diagram showing an example of the structure of coded data and NAL units related to Implementation Example 4.

[0056] Fig.39 This is a diagram showing a semantic example of pcc_nal_unit_type related to Implementation Example 4.

[0057] Fig.40This is a diagram showing an example of the order in which NAL units are sent according to the fourth embodiment.

[0058] Fig.41 This is a flowchart of the processing performed by the three-dimensional data encoding device according to the fourth embodiment.

[0059] Fig.42 This is a flowchart of the processing performed by the three-dimensional data decoding device according to the fourth embodiment.

[0060] Fig.43 This is a flowchart of the multiplexing process related to implementation mode 4.

[0061] Fig.44 This is a flowchart of the inverse multiplexing processing related to implementation mode 4.

[0062] Fig.45 This is a flowchart of the processing performed by the three-dimensional data encoding device according to the fourth embodiment.

[0063] Fig.46 This is a flowchart of the processing performed by the three-dimensional data decoding device according to the fourth embodiment.

[0064] Fig.47 This is a diagram showing an example of dividing slices and tiles according to the fifth embodiment.

[0065] Fig.48 This is a diagram showing an example of a partitioning pattern of slices and tiles according to the fifth embodiment.

[0066] Fig.49 This is a block diagram of the first encoding unit related to implementation mode 6.

[0067] Fig.50 This is a block diagram of the first decoding unit related to implementation mode 6.

[0068] Fig.51 This is a diagram showing an example of the shape of tiles according to the sixth embodiment.

[0069] Fig.52 This is a diagram showing examples of tiles and slices according to the sixth embodiment.

[0070] Fig.53 This is a block diagram of the division unit related to implementation mode 6.

[0071] Fig.54 This is a diagram showing an example of a map showing point cloud data viewed from above according to the sixth embodiment.

[0072] Fig.55 This is a diagram showing an example of tile division according to the sixth embodiment.

[0073] Fig.56This is a diagram showing an example of tile division according to the sixth embodiment.

[0074] Fig.57 This is a diagram showing an example of tile division according to the sixth embodiment.

[0075] Fig.58 This is a diagram showing an example of tile data stored in a server according to the sixth embodiment.

[0076] Fig.59 This is a diagram showing a system regarding tile division according to the sixth embodiment.

[0077] Fig.60 This is a diagram showing an example of slice division according to the sixth embodiment.

[0078] Fig.61 This is a diagram showing an example of dependency relationship related to implementation mode 6.

[0079] Fig.62 This is a diagram showing an example of the decoding order of data related to Implementation Example 6.

[0080] Fig.63 This is a diagram showing an example of coded data of tiles according to the sixth embodiment.

[0081] Fig.64 It is a block diagram showing the combining part related to the sixth embodiment.

[0082] Fig.65 This is a diagram showing an example of the structure of coded data and NAL units related to Implementation Example 6.

[0083] Fig.66 This is a flowchart of the encoding process related to implementation mode 6.

[0084] Fig.67 This is a flowchart of the decoding process related to implementation mode 6.

[0085] Fig.68 This is a diagram showing a syntax example of tile additional information related to Implementation Example 6.

[0086] Fig.69 This is a block diagram of a coding and decoding system related to implementation mode 6.

[0087] Fig.70 This is a diagram showing a syntax example of slice additional information related to implementation mode 6.

[0088] Fig.71 This is a flowchart of the encoding process related to implementation mode 6.

[0089] Fig.72 This is a flowchart of the decoding process related to implementation mode 6.

[0090] Fig.73 This is a flowchart of the encoding process related to implementation mode 6.

[0091] Fig.74 This is a flowchart of the decoding process related to implementation mode 6. DETAILED DESCRIPTION

[0092] A three-dimensional data encoding method according to a technical solution of the present invention generates a plurality of encoded data by encoding a plurality of subspaces formed by dividing an object space including a plurality of three-dimensional points; and generates a bit stream including the plurality of encoded data and the first information representing the shapes of the plurality of subspaces.

[0093] Therefore, the three-dimensional data encoding method can select an arbitrary shape from a plurality of types of subspace shapes, so the encoding efficiency can be improved.

[0094] For example, the shape may be a two-dimensional shape or a three-dimensional shape of the plurality of subspaces.

[0095] For example, the above shape may also be a rectangle or a circle.

[0096] For example, the bit stream may include second information indicating whether the plurality of sub-intervals overlap.

[0097] Therefore, the three-dimensional data encoding method can repeat the subspace, so the subspace can be generated without complicating the shape of the subspace.

[0098] For example, the bit stream may include third information indicating whether the division method of the plurality of sub-intervals is a division method using a bird's-eye view.

[0099] For example, the bit stream may include fourth information indicating at least one of the height, width, depth, and radius of the plurality of sub-intervals.

[0100] For example, the bit stream may include fifth information indicating the position of each of the plurality of sub-intervals.

[0101] For example, the bit stream may include sixth information indicating the number of the plurality of sub-intervals.

[0102] For example, the bit stream may include seventh information indicating intervals between the plurality of subintervals.

[0103] A three-dimensional data decoding method according to a technical solution of the present invention restores the multiple subspaces by decoding multiple encoded data generated by encoding multiple subspaces formed by dividing an object space including multiple three-dimensional points and contained in a bit stream; and restores the object space by combining the multiple subspaces using first information representing shapes of the multiple subspaces contained in the bit stream.

[0104] Therefore, the three-dimensional data decoding method can decode a bit stream with improved coding efficiency that is generated by selecting an arbitrary shape from a plurality of types of subspace shapes.

[0105] For example, the shape may be a two-dimensional shape or a three-dimensional shape of the plurality of subspaces.

[0106] For example, the above shape may also be a rectangle or a circle.

[0107] For example, the bit stream may include second information indicating whether the plurality of sub-intervals overlap; and in restoring the target space, the plurality of sub-spaces may be combined using the second information.

[0108] For example, the bit stream may include third information indicating whether the method for dividing the plurality of subintervals is a method for dividing the plurality of subintervals using ...spaces using a method for dividing the plurality of subspaces using the first information.

[0109] For example, the bit stream may include fourth information indicating at least one of the height, width, depth and radius of the plurality of sub-intervals; and in the restoration of the object space, the fourth information may be used to combine the plurality of sub-spaces.

[0110] For example, the bit stream may include fifth information indicating the position of each of the plurality of sub-intervals; and in the restoration of the object space, the plurality of sub-spaces may be combined using the fifth information.

[0111] For example, the bit stream may include sixth information indicating the number of the plurality of sub-intervals; and in the restoration of the object space, the plurality of sub-spaces may be combined using the sixth information.

[0112] For example, the bit stream may include seventh information indicating intervals between the plurality of sub-intervals; and in the restoration of the object space, the plurality of sub-spaces may be combined using the seventh information.

[0113] In addition, a three-dimensional data encoding device according to a technical solution of the present invention comprises a processor and a memory; the processor uses the memory to perform the following processing: generating multiple encoded data by encoding multiple subspaces formed by dividing an object space including multiple three-dimensional points; and generating a bit stream including the multiple encoded data and the first information representing the shapes of the multiple subspaces.

[0114] Therefore, the three-dimensional data encoding device can select an arbitrary shape from a plurality of types of subspace shapes, thereby improving encoding efficiency.

[0115] In addition, a three-dimensional data decoding device according to a technical solution of the present invention comprises a processor and a memory; the processor uses the memory to perform the following processing: restoring the multiple subspaces by decoding multiple encoded data contained in a bit stream and generated by encoding multiple subspaces formed by dividing an object space including multiple three-dimensional points; and restoring the object space by combining the multiple subspaces using the first information representing the shapes of the multiple subspaces contained in the bit stream.

[0116] Thus, the three-dimensional data decoding device can decode a bit stream with improved coding efficiency that is generated by selecting an arbitrary shape from a plurality of types of subspace shapes.

[0117] In addition, these inclusive or specific technical solutions may also be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0118] Hereinafter, the embodiments are described in detail with reference to the accompanying drawings. In addition, the embodiments described below all represent a specific example of the present invention. The numerical values, shapes, materials, constituent elements, configuration positions and connection forms of constituent elements, steps, and the order of steps shown in the following embodiments are examples and are not intended to limit the present invention. In addition, among the constituent elements of the following embodiments, constituent elements that are not described in the independent claims representing the highest concept are described as arbitrary constituent elements.

[0119] (Implementation Method 1)

[0120] When point cloud coded data is used in actual devices or services, it is desirable to transmit and receive required information according to the application in order to reduce network bandwidth. However, such a function does not exist in the existing 3D data coding structure, and therefore there is no corresponding coding method.

[0121] In this embodiment, a three-dimensional data encoding method and a three-dimensional data encoding device are described for providing the function of sending and receiving required information according to the purpose in the encoded data of a three-dimensional point cloud, a three-dimensional data decoding method and a three-dimensional data decoding device are described for decoding the encoded data, a three-dimensional data multiplexing method for multiplexing the encoded data, and a three-dimensional data transmission method for transmitting the encoded data.

[0122] In particular, currently, the first encoding method and the second encoding method are studied as encoding methods (encoding methods) for point group data, but the composition of the encoded data and the method of saving the encoded data in the system format are not defined. In this case, there is a problem that MUX processing (multiplexing) in the encoding unit, or transmission or accumulation cannot be directly performed.

[0123] Furthermore, there is no method that supports a format in which two codecs, namely the first encoding method and the second encoding method, exist in a mixed form, such as PCC (Point Cloud Compression).

[0124] In this embodiment, a method of configuring PCC coded data in which two codecs, namely, a first coding method and a second coding method, are mixed and storing the coded data in a system format is described.

[0125] First, the configuration of the three-dimensional data (point cloud data) encoding and decoding system according to the present embodiment will be described. Figure 1 2 is a diagram showing an example of the configuration of a three-dimensional data encoding and decoding system according to the present embodiment. Figure 1 As shown, the three-dimensional data encoding and decoding system includes a three-dimensional data encoding system 4601, a three-dimensional data decoding system 4602, a sensor terminal 4603 and an external connection unit 4604.

[0126] The three-dimensional data encoding system 4601 generates encoded data or multiplexed data by encoding point group data as three-dimensional data. In addition, the three-dimensional data encoding system 4601 can be a three-dimensional data encoding device implemented by a single device, or a system implemented by multiple devices. In addition, the three-dimensional data encoding device can also be included in a part of the multiple processing units included in the three-dimensional data encoding system 4601.

[0127] The three-dimensional data encoding system 4601 includes a point cloud data generating system 4611, a presentation unit 4612, an encoding unit 4613, a multiplexing unit 4614, an input / output unit 4615, and a control unit 4616. The point cloud data generating system 4611 includes a sensor information acquiring unit 4617 and a point cloud data generating unit 4618.

[0128] The sensor information acquisition unit 4617 acquires sensor information from the sensor terminal 4603 and outputs the sensor information to the point cloud data generation unit 4618. The point cloud data generation unit 4618 generates point cloud data based on the sensor information and outputs the point cloud data to the encoding unit 4613.

[0129] The presentation unit 4612 presents the sensor information or point group data to the user. For example, the presentation unit 4612 displays information or an image based on the sensor information or point group data.

[0130] The encoding unit 4613 encodes (compresses) the point cloud data, and outputs the obtained encoded data, control information obtained in the encoding process, and other additional information to the multiplexing unit 4614. The additional information includes, for example, sensor information.

[0131] The multiplexing unit 4614 generates multiplexed data by multiplexing the coded data, control information, and additional information input from the coding unit 4613. The format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.

[0132] The input / output unit 4615 (e.g., a communication unit or an interface) outputs the multiplexed data to the outside. Alternatively, the multiplexed data is stored in a storage unit such as an internal memory. The control unit 4616 (or an application execution unit) controls each processing unit. That is, the control unit 4616 performs control such as encoding and multiplexing.

[0133] Furthermore, the sensor information may be input to the encoding unit 4613 or the multiplexing unit 4614. Furthermore, the input / output unit 4615 may directly output the point cloud data or the encoded data to the outside.

[0134] The transmission signal (multiplexed data) output from the three-dimensional data encoding system 4601 is input to the three-dimensional data decoding system 4602 via the external connection unit 4604 .

[0135] The three-dimensional data decoding system 4602 generates point group data as three-dimensional data by decoding the coded data or the multiplexed data. In addition, the three-dimensional data decoding system 4602 may be a three-dimensional data decoding device implemented by a single device, or may be a system implemented by multiple devices. In addition, the three-dimensional data decoding device may also include a part of the multiple processing units included in the three-dimensional data decoding system 4602.

[0136] The three-dimensional data decoding system 4602 includes a sensor information acquisition unit 4621 , an input / output unit 4622 , an inverse multiplexing unit 4623 , a decoding unit 4624 , a prompting unit 4625 , a user interface 4626 , and a control unit 4627 .

[0137] The sensor information acquisition unit 4621 acquires sensor information from the sensor terminal 4603 .

[0138] The input / output unit 4622 obtains the transmission signal, decodes the multiplexed data (file format or packet) according to the transmission signal, and outputs the multiplexed data to the demultiplexing unit 4623 .

[0139] The inverse multiplexing unit 4623 obtains the encoded data, control information, and additional information from the multiplexed data, and outputs the encoded data, control information, and additional information to the decoding unit 4624 .

[0140] The decoding unit 4624 reconstructs the point cloud data by decoding the encoded data.

[0141] The prompt unit 4625 prompts the user with the point group data. For example, the prompt unit 4625 displays information or images based on the point group data. The user interface 4626 obtains instructions based on the user's operation. The control unit 4627 (or the application execution unit) controls each processing unit. That is, the control unit 4627 controls demultiplexing, decoding, and prompting.

[0142] In addition, the input / output unit 4622 may directly obtain point group data or coded data from the outside. In addition, the prompt unit 4625 may obtain additional information such as sensor information and prompt information based on the additional information. In addition, the prompt unit 4625 may also perform prompts based on the user's instructions obtained by the user interface 4626.

[0143] The sensor terminal 4603 generates sensor information, which is information acquired by a sensor. The sensor terminal 4603 is a terminal equipped with a sensor or a camera, and includes, for example, a moving object such as a car, a flying object such as an airplane, a mobile terminal or a camera.

[0144] The sensor information that can be obtained by the sensor terminal 4603 is, for example, (1) the distance between the sensor terminal 4603 and the object, or the reflectivity of the object, obtained by LIDAR, millimeter wave radar, or infrared sensor, (2) the distance between the camera and the object, or the reflectivity of the object, obtained from multiple monocular camera images or stereo camera images. In addition, the sensor information may also include the posture, direction, rotation (angular velocity), position (GPS information or altitude), speed or acceleration of the sensor. In addition, the sensor information may also include temperature, air pressure, humidity, or magnetism.

[0145] The external connection unit 4604 is implemented by an integrated circuit (LSI or IC), an external storage unit, communication with a cloud server via the Internet, or broadcasting.

[0146] Next, point group data will be described. Figure 2 It is a diagram showing the structure of point group data. Figure 3 It is a diagram showing a configuration example of a data file that describes information on point group data.

[0147] Point group data includes data of multiple points. The data of each point includes position information (three-dimensional coordinates) and attribute information corresponding to the position information. A group of multiple such points is called a point group. For example, a point group represents the three-dimensional shape of an object.

[0148] Sometimes, position information (Position) such as three-dimensional coordinates is also called geometry (Geometry). In addition, the data of each point can also include attribute information (attribute) of multiple attribute categories. Attribute categories are, for example, color or reflectivity.

[0149] One piece of attribute information may be associated with one piece of location information, or a plurality of pieces of attribute information having different attribute categories may be associated with one piece of location information. In addition, a plurality of pieces of attribute information having the same attribute category may be associated with one piece of location information.

[0150] Figure 3 The illustrated data file configuration example is an example of a case where position information and attribute information correspond to each other on a one-to-one basis, and indicates position information and attribute information of N points constituting point cloud data.

[0151] The position information is, for example, information on three axes, x, y, and z. The attribute information is, for example, color information of RGB. Representative data files include ply files and the like.

[0152] Next, types of point cloud data will be described. Figure 4 is a graph showing the types of point group data. Figure 4 As shown, the point group data includes static objects and dynamic objects.

[0153] A static object is a 3D point group data at any time (a certain moment). A dynamic object is a 3D point group data that changes over time. Hereinafter, the 3D point group data at a certain moment is referred to as a PCC frame or frame.

[0154] The object may be a point group whose area is limited to a certain extent, such as normal image data, or a large-scale point group whose area is not limited, such as map information.

[0155] In addition, there are point group data of various densities, and there may be sparse point group data and dense point group data.

[0156] The details of each processing unit are described below. The sensor information is obtained by various methods such as a distance sensor such as LIDAR or a rangefinder, a stereo camera, or a combination of multiple monocular cameras. The point group data generation unit 4618 generates point group data based on the sensor information obtained by the sensor information acquisition unit 4617. The point group data generation unit 4618 generates position information as point group data and adds attribute information for the position information to the position information.

[0157] The point group data generation unit 4618 may also process the point group data when generating the position information or the additional attribute information. For example, the point group data generation unit 4618 may also reduce the amount of data by deleting point groups with repeated positions. In addition, the point group data generation unit 4618 may also transform the position information (position conversion, rotation or standardization, etc.) and may also render the attribute information.

[0158] In addition, Figure 1 In the figure, the point group data generating system 4611 is included in the three-dimensional data encoding system 4601, but can also be independently set outside the three-dimensional data encoding system 4601.

[0159] The coding unit 4613 encodes the point cloud data based on a predetermined coding method, thereby generating coded data. There are generally two types of coding methods. The first type is a coding method using position information, which is hereinafter referred to as the first coding method. The second type is a coding method using a video codec, which is hereinafter referred to as the second coding method.

[0160] The decoding unit 4624 decodes the encoded data based on a predetermined encoding method, thereby decoding the point cloud data.

[0161] The multiplexing unit 4614 generates multiplexed data by multiplexing the coded data using an existing multiplexing method. The generated multiplexed data is transmitted or stored. In addition to the PCC coded data, the multiplexing unit 4614 also multiplexes other media such as images, sounds, subtitles, applications, files, or reference time information. In addition, the multiplexing unit 4614 can also multiplex attribute information associated with sensor information or point group data.

[0162] As multiplexing methods or file formats, there are ISOBMFF, MPEG-DASH which is a transmission method based on ISOBMFF, MMT, MPEG-2TS Systems, RMP, etc.

[0163] The demultiplexing unit 4623 extracts PCC coded data, other media, time information, etc. from the multiplexed data.

[0164] The input / output unit 4615 transmits the multiplexed data using a method consistent with a transmission medium or storage medium such as broadcasting or communication. The input / output unit 4615 can communicate with other devices via the Internet, or can communicate with a storage unit such as a cloud server.

[0165] As the communication protocol, http, ftp, TCP, UDP, etc. can be used. Either a PULL type communication method or a PUSH type communication method can be used.

[0166] Any of wired transmission and wireless transmission can be used. As wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), or coaxial cable, etc. are used. As wireless transmission, wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), or millimeter wave, etc. are used.

[0167] In addition, as a broadcasting method, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 is used.

[0168] Figure 5 This diagram shows the configuration of a first coding unit 4630 which is an example of the coding unit 4613 that performs coding using the first coding method. Figure 6 4 is a block diagram of the first coding unit 4630. The first coding unit 4630 generates coded data (coded stream) by coding the point cloud data using the first coding method. The first coding unit 4630 includes a position information coding unit 4631, an attribute information coding unit 4632, an additional information coding unit 4633, and a multiplexing unit 4634.

[0169] The first coding unit 4630 is characterized in that the coding is performed with awareness of the three-dimensional structure. In addition, the first coding unit 4630 is characterized in that the attribute information coding unit 4632 performs coding using information obtained from the position information coding unit 4631. The first coding method is also called GPCC (Geometry based PCC).

[0170] The point group data is PCC point group data such as a PLY file, or PCC point group data generated based on sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData). The position information is input to the position information encoding unit 4631, the attribute information is input to the attribute information encoding unit 4632, and the additional information is input to the additional information encoding unit 4633.

[0171] The position information encoding unit 4631 generates encoded position information (Compressed Geometry) as encoded data by encoding the position information. For example, the position information encoding unit 4631 uses an N-ary tree structure such as an octree to encode the position information. Specifically, in the octree, the object space is divided into 8 nodes (subspaces), and 8 bits of information (occupancy code) are generated to indicate whether each node contains a point group. In addition, the node containing the point group is further divided into 8 nodes, and 8 bits of information are generated to indicate whether each of the 8 nodes contains a point group. This process is repeated until it becomes below the threshold of the number of point groups contained in a predetermined layer or node.

[0172] The attribute information encoding unit 4632 generates the encoded attribute information (Compressed Attribute) as the encoded data by encoding using the composition information generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 determines the reference point (reference node) to be referred to in the encoding of the object point (object node) of the processing object based on the octree structure generated by the position information encoding unit 4631. For example, the attribute information encoding unit 4632 refers to a node whose parent node in the octree is the same as the parent node of the object node among the surrounding nodes or adjacent nodes. In addition, the method of determining the reference relationship is not limited to this.

[0173] In addition, the encoding process of the attribute information may include at least one of a quantization process, a prediction process, and an arithmetic coding process. In this case, the reference means using the reference node in the calculation of the predicted value of the attribute information, or using the state of the reference node in the determination of the encoding parameter (for example, indicating whether the occupancy information of the point group is included in the reference node). For example, the encoding parameter is a quantization parameter in the quantization process, or a context in the arithmetic coding, etc.

[0174] The additional information encoding unit 4633 generates encoded additional information (Compressed MetaData) as encoded data by encoding compressible data in the additional information.

[0175] The multiplexing unit 4634 generates a coded stream (Compressed Stream) as coded data by multiplexing the coding position information, the coding attribute information, the coding additional information, and other additional information. The generated coded stream is output to a processing unit of the system layer (not shown).

[0176] Next, the first decoding unit 4640 which is an example of the decoding unit 4624 that performs decoding according to the first encoding method is described. Figure 7 This is a diagram showing the structure of the first decoding unit 4640. Figure 846 is a block diagram of a first decoding unit 4640. The first decoding unit 4640 generates point cloud data by decoding coded data (coded stream) coded by the first coding method by the first coding method. The first decoding unit 4640 includes a demultiplexing unit 4641, a position information decoding unit 4642, an attribute information decoding unit 4643, and an additional information decoding unit 4644.

[0177] A coded stream (Compressed Stream) as coded data is input to the first decoding unit 4640 from a processing unit of a system layer (not shown).

[0178] The inverse multiplexing unit 4641 separates the encoded position information (Compressed Geometry), the encoded attribute information (Compressed Attribute), the encoded additional information (Compressed MetaData), and other additional information from the encoded data.

[0179] The position information decoding unit 4642 generates position information by decoding the encoded position information. For example, the position information decoding unit 4642 restores the position information of the point group represented by three-dimensional coordinates based on the encoded position information represented by an N-ary tree structure such as an octree.

[0180] The attribute information decoding unit 4643 decodes the encoded attribute information based on the composition information generated by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 determines a reference point (reference node) to be referred to in decoding the object point (object node) of the processing object based on the octree structure obtained by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 refers to a node whose parent node in the octree is the same as the parent node of the object node among the surrounding nodes or adjacent nodes. In addition, the method of determining the reference relationship is not limited to this.

[0181] In addition, the decoding process of the attribute information may also include at least one of an inverse quantization process, a prediction process, and an arithmetic decoding process. In this case, the reference means that the reference node is used in the calculation of the predicted value of the attribute information, or the state of the reference node is used in the determination of the decoded parameter (for example, indicating whether the occupancy information of the point group is included in the reference node). For example, the decoded parameter is a quantization parameter in the inverse quantization process, or a context in the arithmetic decoding, etc.

[0182] The additional information decoding unit 4644 generates additional information by decoding the encoded additional information. In addition, the first decoding unit 4640 uses the additional information required for decoding processing of the position information and the attribute information during decoding, and outputs the additional information required for the application to the outside.

[0183] Next, the second encoding unit 4650 which is an example of the encoding unit 4613 that performs encoding using the second encoding method is described. Fig. 9 This is a diagram showing the structure of the second encoding unit 4650. Fig.10 This is a block diagram of the second encoding unit 4650.

[0184] The second coding unit 4650 generates coded data (coded stream) by coding the point cloud data using the second coding method. The second coding unit 4650 includes an additional information generating unit 4651 , a position image generating unit 4652 , an attribute image generating unit 4653 , a video coding unit 4654 , an additional information coding unit 4655 , and a multiplexing unit 4656 .

[0185] The second coding unit 4650 generates a position image and an attribute image by projecting a three-dimensional structure onto a two-dimensional image, and codes the generated position image and attribute image using an existing video coding method. The second coding method is also called VPCC (video based PCC).

[0186] The point group data is PCC point group data such as a PLY file or PCC point group data generated based on sensor information, and includes position information (Position), attribute information (Attribute), and other additional information (MetaData).

[0187] The additional information generating unit 4651 generates mapping information of a plurality of two-dimensional images by projecting a three-dimensional structure onto a two-dimensional image.

[0188] The position image generation unit 4652 generates a position image (Geometry Image) based on the position information and the mapping information generated by the additional information generation unit 4651. The position image is, for example, a distance image representing the distance (Depth) as a pixel value. In addition, the distance image can be an image of multiple point groups observed from one viewpoint (an image of multiple point groups projected on one two-dimensional plane), or multiple images of multiple point groups observed from multiple viewpoints, or a single image formed by merging these multiple images.

[0189] The attribute image generation unit 4653 generates an attribute image based on the attribute information and the mapping information generated by the additional information generation unit 4651. The attribute image is, for example, an image that represents the attribute information (e.g., color (RGB)) as a pixel value. In addition, the image may be an image of a plurality of point groups observed from a single viewpoint (an image in which a plurality of point groups are projected on a single two-dimensional plane), or may be a plurality of images of a plurality of point groups observed from a plurality of viewpoints, or may be a single image formed by merging these plurality of images.

[0190] The image encoding unit 4654 encodes the position image and the attribute image using an image encoding method, thereby generating a compressed position image (Compressed Geometry Image) and a compressed attribute image (Compressed Attribute Image) as encoded data. In addition, as the image encoding method, any known encoding method can be used. For example, the image encoding method is AVC or HEVC.

[0191] The additional information encoding unit 4655 generates coded additional information (Compressed MetaData) by encoding the additional information and mapping information included in the point cloud data.

[0192] The multiplexing unit 4656 generates a coded stream (Compressed Stream) as coded data by multiplexing the coding position image, the coding attribute image, the coding additional information, and other additional information. The generated coded stream is output to a processing unit of the system layer (not shown).

[0193] Next, the second decoding unit 4660 which is an example of the decoding unit 4624 that performs decoding according to the second encoding method is described. Fig.11 This is a diagram showing the structure of the second decoding unit 4660. Fig.12 46 is a block diagram of a second decoding unit 4660. The second decoding unit 4660 generates point cloud data by decoding coded data (coded stream) coded by the second coding method by the second coding method. The second decoding unit 4660 includes an inverse multiplexing unit 4661, a video decoding unit 4662, an additional information decoding unit 4663, a position information generating unit 4664, and an attribute information generating unit 4665.

[0194] A coded stream (Compressed Stream) as coded data is input to the second decoding unit 4660 from a processing unit of a system layer (not shown).

[0195] The inverse multiplexing unit 4661 separates the coded position image (Compressed Geometry Image), the coded attribute image (Compressed Attribute Image), the coded additional information (Compressed MetaData), and other additional information from the coded data.

[0196] The video decoding unit 4662 generates a position image and an attribute image by decoding the encoded position image and the encoded attribute image using a video encoding method. In addition, as the video encoding method, any known encoding method can be used. For example, the video encoding method is AVC or HEVC.

[0197] The additional information decoding unit 4663 generates additional information including mapping information and the like by decoding the encoded additional information.

[0198] The position information generating unit 4664 generates the position information using the position image and the mapping information. The attribute information generating unit 4665 generates the attribute information using the attribute image and the mapping information.

[0199] The second decoding unit 4660 uses the additional information required for decoding during decoding, and outputs the additional information required for the application to the outside.

[0200] The following describes the issues in the PCC coding method. Fig.13 It is a diagram showing a protocol stack related to PCC coded data. Fig.13 This shows an example of multiplexing, transmitting, or accumulating data of other media such as images (for example, HEVC) or audio into PCC coded data.

[0201] The multiplexing method and file format have functions for multiplexing, transmitting or accumulating various coded data. In order to transmit or accumulate coded data, the coded data must be converted into a format of the multiplexing method. For example, HEVC stipulates a technology for storing coded data in a data structure called a NAL unit and storing the NAL unit in ISOBMFF.

[0202] On the other hand, currently, as a coding method for point group data, the first coding method (Codec1) and the second coding method (Codec2) are being studied, but the composition of the coded data and the method of saving the coded data in the system format are not defined, and there is a problem that MUX processing (multiplexing), transmission and accumulation in the coding unit cannot be directly performed.

[0203] In the following, unless a specific encoding method is mentioned, either the first encoding method or the second encoding method is indicated.

[0204] The following is an explanation of the method for defining the NAL unit of the present embodiment. For example, in previous codecs such as HEVC, a NAL unit of one format is defined for one codec. However, there is no method to support a mixed format of two codecs (hereinafter referred to as PCC codecs) such as the first coding method and the second coding method, as in PCC.

[0205] First, the encoding unit 4670 having the functions of both the first encoding unit 4630 and the second encoding unit 4650 described above, and the decoding unit 4680 having the functions of both the first decoding unit 4640 and the second decoding unit 4660 are described.

[0206] Fig.144 is a block diagram of a coding unit 4670 according to the present embodiment. The coding unit 4670 includes the first coding unit 4630 and the second coding unit 4650 described above, and a multiplexing unit 4671. The multiplexing unit 4671 multiplexes the coded data generated by the first coding unit 4630 and the coded data generated by the second coding unit 4650, and outputs the obtained coded data.

[0207] Fig.15 This is a block diagram of a decoding unit 4680 related to the present embodiment. The decoding unit 4680 includes the first decoding unit 4640 and the second decoding unit 4660 described above and an inverse multiplexing unit 4681. The inverse multiplexing unit 4681 extracts the coded data using the first coding method and the coded data using the second coding method from the input coded data. The inverse multiplexing unit 4681 outputs the coded data using the first coding method to the first decoding unit 4640, and outputs the coded data using the second coding method to the second decoding unit 4660.

[0208] With the above configuration, the encoding unit 4670 can selectively encode the point cloud data using the first encoding method and the second encoding method. In addition, the decoding unit 4680 can decode the encoded data encoded using the first encoding method, the encoded data encoded using the second encoding method, and the encoded data encoded using both the first encoding method and the second encoding method.

[0209] For example, the coding unit 4670 may switch the coding method (the first coding method and the second coding method) in units of point cloud data or frames. In addition, the coding unit 4670 may switch the coding method in units of codable data.

[0210] The encoding unit 4670 generates, for example, encoded data (encoded stream) including identification information of the PCC codec.

[0211] The demultiplexing unit 4681 included in the decoding unit 4680 identifies the data using identification information of the PCC codec, for example. The demultiplexing unit 4681 outputs the data to the first decoding unit 4640 if the data is data encoded using the first encoding method, and outputs the data to the second decoding unit 4660 if the data is data encoded using the second encoding method.

[0212] Furthermore, the coding unit 4670 may send information indicating whether both coding methods or one of the coding methods is used as control information, in addition to the identification information of the PCC codec.

[0213] Next, the encoding process according to this embodiment will be described. Fig.16 2 is a flowchart of the encoding process according to the present embodiment. By using the identification information of the PCC codec, encoding processes corresponding to a plurality of codecs can be performed.

[0214] First, the encoding unit 4670 encodes the PCC data using a codec of one or both of the first encoding method and the second encoding method (S4681).

[0215] When the codec used is the second encoding method (the second encoding method in S4682), the encoder 4670 sets the pcc_codec_type included in the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded using the second encoding method (S4683). Next, the encoder 4670 sets the identifier of the NAL unit used for the second encoding method for the pcc_nal_unit_type of the NAL unit header (S4684). The encoder 4670 generates a NAL unit having the set NAL unit header and including the encoded data in the payload. The encoder 4670 transmits the generated NAL unit (S4685).

[0216] On the other hand, when the codec used is the first encoding method (the first encoding method in S4682), the encoder 4670 sets the pcc_codec_type included in the NAL unit header to a value indicating that the data included in the payload of the NAL unit is data encoded using the first encoding method (S4686). Next, the encoder 4670 sets the identifier of the NAL unit used for the first encoding method for the pcc_nal_unit_type of the NAL unit header (S4687). Next, the encoder 4670 generates a NAL unit having the set NAL unit header and including the encoded data in the payload. And, the encoder 4670 sends the generated NAL unit (S4685).

[0217] Next, the decoding process according to this embodiment will be described. Fig.17 This is a flowchart of the decoding process according to the present embodiment. By using the identification information of the PCC codec, it is possible to perform decoding processes corresponding to a plurality of codecs.

[0218] First, the decoding unit 4680 receives a NAL unit (S4691). For example, the NAL unit is generated by the process in the encoding unit 4670 described above.

[0219] Next, the decoding unit 4680 determines whether pcc_codec_type included in the NAL unit header indicates the first encoding method or the second encoding method ( S4692 ).

[0220] When pcc_codec_type indicates the second coding method (the second coding method in S4692), the decoding unit 4680 determines that the data included in the payload of the NAL unit is data encoded using the second coding method (S4693). Furthermore, the second decoding unit 4660 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the second coding method (S4694). Furthermore, the decoding unit 4680 decodes the PCC data using the decoding process of the second coding method (S4695).

[0221] On the other hand, when pcc_codec_type indicates the first coding method (the first coding method in S4692), the decoding unit 4680 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4696). Furthermore, the decoding unit 4680 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first coding method (S4697). Furthermore, the decoding unit 4680 decodes the PCC data using the decoding process of the first coding method (S4698).

[0222] As described above, a three-dimensional data encoding device according to a technical solution of the present invention generates a coding stream by encoding three-dimensional data (e.g., point group data), and stores information indicating a coding method used in the above coding, among the first coding method and the second coding method, in control information (e.g., a parameter set) of the above coding stream (e.g., identification information of a codec).

[0223] Thus, when decoding the coded stream generated by the 3D data coding device, the 3D data decoding device can use the information stored in the control information to determine the coding method used in the coding. Therefore, the 3D data decoding device can correctly decode the coded stream even when multiple coding methods are used.

[0224] For example, the three-dimensional data includes position information. The three-dimensional data encoding device encodes the position information in the encoding. The three-dimensional data encoding device stores information indicating the encoding method used in encoding the position information in the control information of the position information in the storage.

[0225] For example, the three-dimensional data includes position information and attribute information. The three-dimensional data encoding device encodes the position information and the attribute information in the encoding. The three-dimensional data encoding device stores information indicating the encoding method used in encoding the position information in the control information of the position information and the encoding method used in encoding the attribute information in the control information of the attribute information.

[0226] This allows different encoding methods to be used for position information and attribute information, thereby improving encoding efficiency.

[0227] For example, the three-dimensional data encoding method further stores the encoding stream in one or more units (eg, NAL units).

[0228] For example, the above-mentioned unit includes the following information (such as pcc_nal_unit_type), which has a common format in the above-mentioned first encoding method and the above-mentioned second encoding method, indicates the type of data included in the above-mentioned unit, and has independent definitions in the above-mentioned first encoding method and the above-mentioned second encoding method.

[0229] For example, the above-mentioned unit includes the following information (such as codec1_nal_unit_type or codec2_nal_unit_type), which has a format independent of the above-mentioned first encoding method and the above-mentioned second encoding method, indicates the type of data included in the above-mentioned unit, and has a definition independent of the above-mentioned first encoding method and the above-mentioned second encoding method.

[0230] For example, the above-mentioned unit includes the following information (such as pcc_nal_unit_type), which has a common format in the above-mentioned first encoding method and the above-mentioned second encoding method, indicates the type of data contained in the above-mentioned unit, and has a common definition in the above-mentioned first encoding method and the above-mentioned second encoding method.

[0231] For example, the three-dimensional data encoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0232] In addition, the three-dimensional data decoding device related to the present embodiment determines the encoding method used in encoding the encoded stream based on information indicating the encoding method used in encoding the three-dimensional data among the first encoding method and the second encoding method (e.g., identification information of the codec) contained in control information (e.g., a parameter set) of the encoded stream generated by encoding the three-dimensional data, and decodes the encoded stream using the determined encoding method.

[0233] Thus, when decoding the coded stream, the 3D data decoding device can use the information stored in the control information to determine the coding method used in the coding. Therefore, the 3D data decoding device can correctly decode the coded stream even when multiple coding methods are used.

[0234] For example, the three-dimensional data includes position information, and the coded stream includes coded data of the position information. In the determination, the three-dimensional data decoding device determines the coding method used in coding the position information based on information indicating the coding method used in coding the position information, of the first coding method and the second coding method, included in the control information of the position information included in the coded stream. In the decoding, the three-dimensional data decoding device decodes the coded data of the position information using the coding method used in coding the determined position information.

[0235] For example, the three-dimensional data includes position information and attribute information, and the coded stream includes coded data of the position information and coded data of the attribute information. In the determination, the three-dimensional data decoding device determines the coding method used in coding the position information based on information indicating the coding method used in coding the position information between the first coding method and the second coding method, which is included in the control information of the position information included in the coded stream, and determines the coding method used in coding the attribute information based on information indicating the coding method used in coding the attribute information between the first coding method and the second coding method, which is included in the control information of the attribute information included in the coded stream. In the decoding, the three-dimensional data decoding device decodes the coded data of the position information using the coding method determined to be used in coding the position information, and decodes the coded data of the attribute information using the coding method determined to be used in coding the attribute information.

[0236] This allows different encoding methods to be used for position information and attribute information, thereby improving encoding efficiency.

[0237] For example, the coded stream is stored in one or more units (for example, NAL units), and the three-dimensional data decoding device further obtains the coded stream from the one or more units.

[0238] For example, the above-mentioned unit includes the following information (such as pcc_nal_unit_type), which has a common format in the above-mentioned first encoding method and the above-mentioned second encoding method, indicates the type of data included in the above-mentioned unit, and has independent definitions in the above-mentioned first encoding method and the above-mentioned second encoding method.

[0239] For example, the above-mentioned unit includes the following information (such as codec1_nal_unit_type or codec2_nal_unit_type), which has an independent format in the above-mentioned first encoding method and the above-mentioned second encoding method, indicates the type of data included in the above-mentioned unit, and has an independent definition in the above-mentioned first encoding method and the above-mentioned second encoding method.

[0240] For example, the above-mentioned unit includes the following information (such as pcc_nal_unit_type), which has a common format in the above-mentioned first encoding method and the above-mentioned second encoding method, indicates the type of data contained in the above-mentioned unit, and has a common definition in the above-mentioned first encoding method and the above-mentioned second encoding method.

[0241] For example, the three-dimensional data decoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0242] (Implementation Method 2)

[0243] In this embodiment, a method of storing a NAL unit in an ISOBMFF file is described.

[0244] ISOBMFF (ISO based media file format) is a file format standard defined by ISO / IEC 14496-12. ISOBMFF defines a format that can multiplex and store various media such as video, audio, and text, and is a standard that is independent of the media.

[0245] The basic structure (file) of ISOBMFF is explained. The basic unit in ISOBMFF is a box. A box consists of type, length, and data. The combination of boxes of various types is called a file.

[0246] Fig.18 This is a diagram showing the basic structure (file) of ISOBMFF. The ISOBMFF file mainly includes boxes such as ftyp which indicates the file version (brand) using 4CC (4 character code), moov which stores metadata such as control information, and mdat which stores data.

[0247] The method of saving each media in the ISOBMFF file is separately specified. For example, the method of saving AVC video and HEVC video is specified by ISO / IEC14496-15. Here, in order to store or transmit PCC coded data, it is conceivable to use the ISOBMFF function extension, but there is no provision for saving PCC coded data in the ISOBMFF file. Therefore, in this embodiment, the method of saving PCC coded data in the ISOBMFF file is described.

[0248] Fig.19 This is a diagram showing a protocol stack when a NAL unit common to a PCC codec is saved in an ISOBMFF file. Here, a NAL unit common to a PCC codec is saved in an ISOBMFF file. NAL units are common to PCC codecs, but since multiple PCC codecs are saved in a NAL unit, it is desirable to define a saving method corresponding to each codec (Carriage of Codec1, Carriage of Codec2).

[0249] Next, a method of storing a common PCC NAL unit supporting a plurality of PCC codecs in an ISOBMFF file is described. Fig. 20 This diagram shows an example of storing a common PCC NAL unit in a file of ISOBMFF of the storage method of codec 1 (Carriage of Codec1). Fig.21 This diagram shows an example of storing a common PCC NAL unit in an ISOBMFF file of the storage method (Carriage of Codec2) of codec 2.

[0250] Here, ftyp is important information for identifying the file format, and a different identifier for each codec is defined as ftyp. When PCC coded data encoded by the first coding method (coding mode) is saved in a file, ftyp=pcc1 is set. When PCC coded data encoded by the second coding method is saved in a file, ftyp=pcc2 is set.

[0251] Here, pcc1 indicates codec 1 using PCC (first encoding method). pcc2 indicates codec 2 using PCC (second encoding method). That is, pcc1 and pcc2 indicate that the data is PCC (coded data of three-dimensional data (point cloud data)), and indicate PCC codec (first encoding method and second encoding method).

[0252] The following describes a method of storing the NAL unit in an ISOBMFF file. The multiplexing unit parses the NAL unit header, and when pcc_codec_type=Codec1, pcc1 is written in the ISOBMFF ftyp.

[0253] Furthermore, the multiplexing unit parses the NAL unit header, and when pcc_codec_type=Codec2, pcc2 is recorded in ftyp of ISOBMFF.

[0254] Furthermore, when pcc_nal_unit_type is metadata, the multiplexing unit stores the NAL unit in, for example, moov or mdat using a predetermined method. When pcc_nal_unit_type is data, the multiplexing unit stores the NAL unit in, for example, moov or mdat using a predetermined method.

[0255] For example, the multiplexing unit may store the NAL unit size in the NAL unit similarly to HEVC.

[0256] By parsing the ftyp contained in the file in the inverse multiplexing unit (system layer) using this storage method, it is possible to determine whether the PCC coded data is encoded using the first coding method or the second coding method. Furthermore, as described above, by determining whether the PCC coded data is encoded using the first coding method or the second coding method, it is possible to extract coded data encoded using one of the coding methods from data in which coded data encoded using both coding methods are mixed. Thus, when transmitting coded data, the amount of data transmitted can be suppressed. In addition, by using this storage method, it is possible to use a common data format instead of setting different data (file) formats in the first coding method and the second coding method.

[0257] In addition, when the identification information of the codec is indicated in metadata of the system layer such as ftyp in ISOBMFF, the multiplexing unit may store the NAL unit after deleting pcc_nal_unit_type in the ISOBMFF file.

[0258] Next, the structure and operation of the multiplexing unit of the three-dimensional data encoding system (three-dimensional data encoding device) according to the present embodiment and the demultiplexing unit of the three-dimensional data decoding system (three-dimensional data decoding device) according to the present embodiment are described.

[0259] Fig. 2247 is a diagram showing the structure of the first multiplexing unit 4710. The first multiplexing unit 4710 includes a file conversion unit 4711 that generates multiplexed data (file) by storing the coded data and control information (NAL unit) generated by the first coding unit 4630 in an ISOBMFF file. The first multiplexing unit 4710 includes, for example, Figure 1 In the multiplexing unit 4614 shown.

[0260] Fig.23 47 is a diagram showing the structure of the first demultiplexing unit 4720. The first demultiplexing unit 4720 includes a file inverse conversion unit 4721 that obtains coded data and control information (NAL unit) from the multiplexed data (file) and outputs the obtained coded data and control information to the first decoding unit 4640. The first demultiplexing unit 4720 includes, for example, Figure 1 In the inverse multiplexing unit 4623 shown.

[0261] Fig.24 4730 is a diagram showing the structure of the second multiplexing unit 4730. The second multiplexing unit 4730 includes a file conversion unit 4731 that generates multiplexed data (file) by storing the coded data and control information (NAL unit) generated by the second coding unit 4650 in an ISOBMFF file. The second multiplexing unit 4730 is included in, for example, Figure 1 In the multiplexing unit 4614 shown.

[0262] Fig.25 The second demultiplexing unit 4740 is a diagram showing the structure of the second demultiplexing unit 4740. The second demultiplexing unit 4740 includes a file inverse conversion unit 4741 that obtains coded data and control information (NAL unit) from the multiplexed data (file) and outputs the obtained coded data and control information to the second decoding unit 4660. The second demultiplexing unit 4740 includes, for example, Figure 1 In the inverse multiplexing unit 4623 shown.

[0263] Fig.26 This is a flowchart of the multiplexing process performed by the first multiplexing unit 4710. First, the first multiplexing unit 4710 analyzes pcc_codec_type included in the NAL unit header to determine whether the codec used is the first encoding method or the second encoding method (S4701).

[0264] When pcc_codec_type indicates the second encoding method (the second encoding method in S4702), the first multiplexing unit 4710 does not process the NAL unit (S4703).

[0265] On the other hand, when pcc_codec_type indicates the second coding method (the first coding method in S4702), the first multiplexer 4710 records pcc1 in ftyp (S4704). That is, the first multiplexer 4710 records information indicating that data encoded by the first coding method is stored in the file in ftyp.

[0266] Next, the first multiplexing unit 4710 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_nal_unit_type (S4705). In addition, the first multiplexing unit 4710 creates an ISOBMFF file including the ftyp and the box (S4706).

[0267] Fig. 27 This is a flowchart of the multiplexing process performed by the second multiplexing unit 4730. First, the second multiplexing unit 4730 analyzes pcc_codec_type included in the NAL unit header to determine whether the codec used is the first encoding method or the second encoding method (S4711).

[0268] When pcc_unit_type indicates the second coding method (the second coding method in S4712), the second multiplexing unit 4730 records pcc2 in ftyp (S4713). That is, the second multiplexing unit 4730 records information indicating that data encoded by the second coding method is stored in the file in ftyp.

[0269] Next, the second multiplexing unit 4730 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_nal_unit_type (S4714). Furthermore, the second multiplexing unit 4730 creates an ISOBMFF file including the ftyp and the box (S4715).

[0270] On the other hand, when pcc_unit_type indicates the first encoding method (the first encoding method in S4712), the second multiplexing unit 4730 does not process the NAL unit (S4716).

[0271] In addition, the above-mentioned process shows an example of encoding PCC data using either the first encoding method or the second encoding method. The first multiplexing unit 4710 and the second multiplexing unit 4730 save the desired NAL unit to the file by identifying the codec type of the NAL unit. In addition, in the case where the identification information of the PCC codec is included in addition to the NAL unit header, the first multiplexing unit 4710 and the second multiplexing unit 4730 may also use the identification information of the PCC codec included in addition to the NAL unit header to identify the codec type (the first encoding method or the second encoding method) in steps S4701 and S4711.

[0272] Furthermore, when the first multiplexing unit 4710 and the second multiplexing unit 4730 save the data in the file in steps S4706 and S4714, they may delete pcc_nal_unit_type from the NAL unit header and save the data in the file.

[0273] Fig.28 This is a flowchart showing the processing performed by the first demultiplexing unit 4720 and the first decoding unit 4640. First, the first demultiplexing unit 4720 parses the ftyp contained in the ISOBMFF file (S4721). In the case where the codec represented by ftyp is the second coding method (pcc2) (it is the second coding method in S4722), the first demultiplexing unit 4720 determines that the data contained in the payload of the NAL unit is data encoded using the second coding method (S4723). In addition, the first demultiplexing unit 4720 passes the determination result to the first decoding unit 4640. The first decoding unit 4640 does not process the NAL unit (S4724).

[0274] On the other hand, when the codec indicated by ftyp is the first coding method (pcc1) (the first coding method in S4722), the first demultiplexing unit 4720 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4725). In addition, the first demultiplexing unit 4720 transmits the determination result to the first decoding unit 4640.

[0275] The first decoding unit 4640 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first encoding method (S4726). Then, the first decoding unit 4640 decodes the PCC data using the decoding process of the first encoding method (S4727).

[0276] Fig.294731. This is a flowchart showing the processing performed by the second demultiplexing unit 4740 and the second decoding unit 4660. First, the second demultiplexing unit 4740 parses the ftyp contained in the ISOBMFF file (S4731). When the codec represented by ftyp is the second coding method (pcc2) (the second coding method in S4732), the second demultiplexing unit 4740 determines that the data contained in the payload of the NAL unit is data encoded using the second coding method (S4733). In addition, the second demultiplexing unit 4740 passes the determination result to the second decoding unit 4660.

[0277] The second decoding unit 4660 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the second encoding method (S4734). Then, the second decoding unit 4660 decodes the PCC data using the decoding process of the second encoding method (S4735).

[0278] On the other hand, when the codec indicated by ftyp is the first coding method (pcc1) (the first coding method in S4732), the second demultiplexing unit 4740 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4736). Furthermore, the second demultiplexing unit 4740 transmits the determination result to the second decoding unit 4660. The second decoding unit 4660 does not process the NAL unit (S4737).

[0279] In this way, for example, by identifying the codec type of the NAL unit in the first demultiplexing unit 4720 or the second demultiplexing unit 4740, the codec type can be identified at an earlier stage. Furthermore, the desired NAL unit can be input to the first decoding unit 4640 or the second decoding unit 4660, and unnecessary NAL units can be removed. In this case, in the first decoding unit 4640 or the second decoding unit 4660, it may be unnecessary to parse the identification information of the codec. In addition, it is also possible to parse the identification information of the codec again by referring to the NAL unit type in the first decoding unit 4640 or the second decoding unit 4660.

[0280] Furthermore, when pcc_nal_unit_type is deleted from the NAL unit header in the first multiplexing unit 4710 or the second multiplexing unit 4730 , the first inverse multiplexing unit 4720 or the second inverse multiplexing unit 4740 may output the NAL unit to the first decoding unit 4640 or the second decoding unit 4660 after assigning pcc_nal_unit_type to the NAL unit.

[0281] (Implementation method 3)

[0282] In this embodiment, a multiplexing unit and a demultiplexing unit corresponding to the encoding unit 4670 and the decoding unit 4680 corresponding to a plurality of codecs described in Embodiment 1 are described. Fig.30 This is a diagram showing the configuration of the encoding unit 4670 and the third multiplexing unit 4750 according to this embodiment.

[0283] The coding unit 4670 codes the point cloud data using one or both of the first coding method and the second coding method. The coding unit 4670 may switch the coding method (the first coding method and the second coding method) in units of point cloud data or frames. In addition, the coding unit 4670 may switch the coding method in units that can be coded.

[0284] The encoding unit 4670 generates encoded data (encoded stream) including identification information of the PCC codec.

[0285] The third multiplexing unit 4750 includes a file conversion unit 4751. The file conversion unit 4751 converts the NAL unit output from the encoding unit 4670 into a file of PCC data. The file conversion unit 4751 parses the codec identification information included in the NAL unit header to determine whether the PCC coded data is data encoded by the first coding method, data encoded by the second coding method, or data encoded by both methods. The file conversion unit 4751 records the version name that can identify the codec in ftyp. For example, in the case of indicating encoding by both methods, pcc3 is recorded in ftyp.

[0286] Furthermore, when the encoding unit 4670 describes the identification information of the PCC codec other than the NAL unit, the file conversion unit 4751 may determine the PCC codec (encoding method) using the identification information.

[0287] Fig.31 This is a diagram showing the configuration of the third inverse multiplexing unit 4760 and the decoding unit 4680 according to the present embodiment.

[0288] The third demultiplexing unit 4760 includes a file inverse conversion unit 4761. The file inverse conversion unit 4761 analyzes ftyp included in the file and determines whether the PCC coded data is data coded using the first coding method, data coded using the second coding method, or data coded using both methods.

[0289] When the PCC coded data is coded using one of the coding methods, the data is input to the corresponding decoding unit of the first decoding unit 4640 and the second decoding unit 4660, and no data is input to the other decoding unit. When the PCC coded data is coded using both coding methods, the data is input to the decoding unit 4680 corresponding to both methods.

[0290] The decoding unit 4680 decodes the PCC coded data using one or both of the first coding method and the second coding method.

[0291] Fig.32 It is a flowchart showing the processing performed by the third multiplexing unit 4750 related to this embodiment.

[0292] First, the third multiplexing unit 4750 analyzes pcc_codec_type included in the NAL unit header to determine whether the codec used is the first encoding method, the second encoding method, or both the first encoding method and the second encoding method (S4741).

[0293] When the second encoding method is used (Yes in S4742, and the second encoding method is used in S4743), the third multiplexing unit 4750 records pcc2 in ftyp (S4744). That is, the third multiplexing unit 4750 records information indicating that data encoded by the second encoding method is stored in the file in ftyp.

[0294] Next, the third multiplexing unit 4750 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_unit_type (S4745). Furthermore, the third multiplexing unit 4750 creates an ISOBMFF file including the ftyp and the box (S4746).

[0295] On the other hand, when the first encoding method is used ("Yes" in S4742, and the first encoding method is used in S4743), the third multiplexing unit 4750 records pcc1 in ftyp (S4747). That is, the third multiplexing unit 4750 records information indicating that data encoded by the first encoding method is stored in the file in ftyp.

[0296] Next, the third multiplexing unit 4750 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_unit_type (S4748). Furthermore, the third multiplexing unit 4750 creates an ISOBMFF file including the ftyp and the box (S4746).

[0297] On the other hand, when both the first encoding method and the second encoding method are used (No in S4742), the third multiplexing unit 4750 records pcc3 in ftyp (S4749). That is, the third multiplexing unit 4750 records information indicating that data encoded by both encoding methods is stored in the file in ftyp.

[0298] Next, the third multiplexing unit 4750 parses the pcc_nal_unit_type included in the NAL unit header, and saves the data into a box (moov or mdat, etc.) using a predetermined method corresponding to the data type indicated by pcc_unit_type (S4750). Furthermore, the third multiplexing unit 4750 creates an ISOBMFF file including the ftyp and the box (S4746).

[0299] Fig.33 4760 and the decoding unit 4680. First, the third demultiplexing unit 4760 parses the ftyp contained in the ISOBMFF file (S4761). In the case where the codec represented by ftyp is the second coding method (pcc2) ("yes" in S4762, and it is the second coding method in S4763), the third demultiplexing unit 4760 determines that the data contained in the payload of the NAL unit is data encoded using the second coding method (S4764). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0300] The decoding unit 4680 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the second encoding method (S4765). Then, the decoding unit 4680 decodes the PCC data using the decoding process of the second encoding method (S4766).

[0301] On the other hand, when the codec indicated by ftyp is the first coding method (pcc1) ("Yes" in S4762, and the first coding method in S4763), the third demultiplexing unit 4760 determines that the data included in the payload of the NAL unit is data encoded using the first coding method (S4767). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0302] The decoding unit 4680 identifies the data by assuming that pcc_nal_unit_type included in the NAL unit header is an identifier of the NAL unit for the first encoding method (S4768). Then, the decoding unit 4680 decodes the PCC data using the decoding process of the first encoding method (S4769).

[0303] On the other hand, when ftyp indicates that both coding methods (pcc3) are used (No in S4762), the third demultiplexing unit 4760 determines that the data included in the payload of the NAL unit is data encoded using both the first coding method and the second coding method (S4770). In addition, the third demultiplexing unit 4760 transmits the determination result to the decoding unit 4680.

[0304] The decoding unit 4680 identifies the data by setting the pcc_nal_unit_type included in the NAL unit header to be the identifier of the NAL unit for the codec described in pcc_codec_type (S4771). Furthermore, the decoding unit 4680 decodes the PCC data using the decoding processing of both encoding methods (S4772). That is, the decoding unit 4680 decodes the data encoded by the first encoding method using the decoding processing of the first encoding method, and decodes the data encoded by the second encoding method using the decoding processing of the second encoding method.

[0305] The following describes a modified example of the present embodiment. As the type of version represented by ftyp, the following types may be represented by identification information. In addition, a combination of the following types may be represented by identification information.

[0306] The identification information indicates whether the object of the original data before PCC encoding is a point group with a limited area or a large-scale point group with an unlimited area like map information.

[0307] The identification information may also indicate whether the original data before PCC encoding is a static object or a dynamic object.

[0308] As described above, the identification information may indicate whether the PCC coded data is data coded using the first coding method or data coded using the second coding method.

[0309] The identification information may indicate an algorithm used in PCC encoding. Here, the algorithm is, for example, an encoding method that can be used in the first encoding method or the second encoding method.

[0310] The identification information may also indicate the difference in the method of storing the PCC coded data in the ISOBMFF file. For example, the identification information may indicate whether the storage method used is a storage method for accumulation or a storage method for real-time transmission such as dynamic streaming.

[0311] In addition, in Embodiments 2 and 3, an example of using ISOBMFF as a file format is described, but other methods may also be used. For example, the same method as in this embodiment may be used when saving PCC encoded data to MPEG-2TS Systems, MPEG-DASH, MMT, or RMP.

[0312] In the above, an example is shown in which metadata such as identification information is stored in ftyp, but these metadata may be stored outside of ftyp. For example, these metadata may be stored in moov.

[0313] As described above, the three-dimensional data storage device (or three-dimensional data multiplexing device, or three-dimensional data encoding device) performs Fig.34 Processing shown.

[0314] First, the three-dimensional data storage device (for example, including the first multiplexing unit 4710, the second multiplexing unit 4730, or the third multiplexing unit 4750) obtains one or more units (for example, NAL units) storing a coded stream obtained by encoding point group data (S4781). Next, the three-dimensional data storage device saves the one or more units to a file (for example, an ISOBMFF file) (S4782). In addition, during the saving (S4782), the three-dimensional data storage device saves information indicating that the data stored in the file is data obtained by encoding point group data (for example, pcc1, pcc2, or pcc3) in the control information (for example, ftyp) of the above file.

[0315] Thus, in a device that processes a file generated by the three-dimensional data storage device, it is possible to refer to the control information of the file and determine at an early stage whether the data stored in the file is coded data of point group data, thereby reducing the processing amount of the device or speeding up the processing.

[0316] For example, the information also indicates the encoding method used in encoding the point group data in the first encoding method and the second encoding method. In addition, the data stored in the file is the data obtained by encoding the point group data, and the encoding method used in encoding the point group data in the first encoding method and the second encoding method, which can be represented by a single information or different information.

[0317] Thus, in a device that processes a file generated by the three-dimensional data storage device, the codec used for the data stored in the file can be determined earlier by referring to the file control information, thereby reducing the amount of processing of the device or speeding up the processing.

[0318] For example, the first encoding method is a method (GPCC) of encoding position information of the point group data represented by an N (N is an integer greater than 2) fork tree and using the position information to encode attribute information, and the second encoding method is a method (VPCC) of generating a two-dimensional image based on the point group data and encoding the two-dimensional image using an image encoding method.

[0319] For example, the above file is based on ISOBMFF (ISO based media file format: ISO base media file format).

[0320] For example, the three-dimensional data storage device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0321] In addition, as described above, the three-dimensional data acquisition device (or three-dimensional data demultiplexing device, or three-dimensional data decoding device) performs Fig.35 Processing shown.

[0322] The three-dimensional data acquisition device (for example, including the first inverse multiplexing unit 4720, the second inverse multiplexing unit 4740, or the third inverse multiplexing unit 4760) acquires a file (for example, an ISOBMFF file) storing one or more units (for example, NAL units), wherein the one or more units store a coded stream obtained by encoding point group data (S4791). Next, the three-dimensional data acquisition device acquires one or more units from the file (S4792). In addition, the control information of the file (for example, ftyp) includes information indicating that the data stored in the file is data obtained by encoding the point group data (for example, pcc1, pcc2, or pcc3).

[0323] For example, the three-dimensional data acquisition device refers to the above information to determine whether the data stored in the file is data obtained by encoding the point group data. In addition, when the three-dimensional data acquisition device determines that the data stored in the file is data obtained by encoding the point group data, it generates the point group data by decoding the data obtained by encoding the point group data contained in one or more units. Alternatively, when the three-dimensional data acquisition device determines that the data stored in the file is data obtained by encoding the point group data, it outputs (notifies) information indicating that the data contained in one or more units is data obtained by encoding the point group data to a subsequent processing unit (for example, the first decoding unit 4640, the second decoding unit 4660, or the decoding unit 4680).

[0324] Thus, the 3D data acquisition device can refer to the control information of the file and determine whether the data stored in the file is the coded data of the point group data at an early stage, thereby reducing the processing amount of the 3D data acquisition device or the subsequent device or speeding up the processing.

[0325] For example, the information also indicates the encoding method used in the encoding of the first encoding method and the second encoding method. In addition, the data stored in the file is the data obtained by encoding the point group data, and the encoding method used in the encoding of the point group data of the first encoding method and the second encoding method, which can be represented by a single information or different information.

[0326] Thus, the 3D data acquisition device can refer to the control information of the file and determine the codec used for the data stored in the file at an early stage, thereby reducing the amount of processing of the 3D data acquisition device or a subsequent device or increasing the processing speed.

[0327] For example, the three-dimensional data acquisition device acquires data encoded by one of the encoding methods from the encoded point cloud data including data encoded by the first encoding method and data encoded by the second encoding method based on the above information.

[0328] For example, the first encoding method is a method (GPCC) of encoding position information of the point group data represented by an N (N is an integer greater than 2) fork tree and using the position information to encode attribute information, and the second encoding method is a method (VPCC) of generating a two-dimensional image based on the point group data and encoding the two-dimensional image using an image coding method.

[0329] For example, the above file is based on ISOBMFF (ISO based media file format).

[0330] For example, the three-dimensional data acquisition device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0331] (Implementation 4)

[0332] In this embodiment, the types of coded data (position information (Geometry), attribute information (Attribute), additional information (Metadata)) generated by the first coding unit 4630 or the second coding unit 4650, the generation method of the additional information (metadata), and the multiplexing process in the multiplexing unit are described. In addition, the additional information (metadata) may be expressed as a parameter set or control information.

[0333] In this embodiment, Figure 4 The dynamic object (three-dimensional point group data that changes with time) described in the above is used as an example, but the same method can also be used in the case of a static object (three-dimensional point group data at an arbitrary time).

[0334] Fig.36The diagram shows the configuration of a coding unit 4801 and a multiplexing unit 4802 included in the three-dimensional data coding device according to the present embodiment. The coding unit 4801 corresponds to, for example, the first coding unit 4630 or the second coding unit 4650 described above. The multiplexing unit 4802 corresponds to the multiplexing unit 4634 or 46456 described above.

[0335] The encoding unit 4801 encodes the point cloud data of a plurality of PCC (Point Cloud Compression) frames to generate encoded data (Multiple Compressed Data) of a plurality of position information, attribute information, and additional information.

[0336] The multiplexing unit 4802 converts the data of a plurality of data types (position information, attribute information, and additional information) into a data structure that takes data access in the decoding device into consideration by converting the data into NAL units.

[0337] Fig.37 4801. The arrows in the figure represent dependency relationships related to the decoding of the coded data, and the root of the arrow depends on the data at the tip of the arrow. That is, the decoding device decodes the data at the tip of the arrow, and uses the decoded data to decode the data at the root of the arrow. In other words, dependency means that the data of the dependency target is referenced (used) in the processing (encoding or decoding, etc.) of the data of the dependency source.

[0338] First, the generation process of the encoded data of the position information is described. The encoding unit 4801 generates the encoded position data (Compressed Geometry Data) of each frame by encoding the position information of each frame. In addition, the encoded position data is represented by G(i). i represents the frame number or the time of the frame, etc.

[0339] In addition, the encoding unit 4801 generates a position parameter set (GPS(i)) corresponding to each frame. The position parameter set includes parameters that can be used in decoding the encoded position data. In addition, the encoded position data of each frame depends on the corresponding position parameter set.

[0340] In addition, the coded position data composed of a plurality of frames is defined as a position sequence (Geometry Sequence). The coding unit 4801 generates a position sequence parameter set (Geometry Sequence PS: also referred to as position SPS) that stores parameters commonly used in decoding processing of a plurality of frames in the position sequence. The position sequence depends on the position SPS.

[0341] Next, the generation process of the encoded data of the attribute information is described. The encoding unit 4801 generates the compressed attribute data (Compressed Attribute Data) of each frame by encoding the attribute information of each frame. In addition, the compressed attribute data is represented by A(i). In addition, Fig.37 , an example in which attribute X and attribute Y exist is shown, and the encoded attribute data of attribute X is represented by AX(i), and the encoded attribute data of attribute Y is represented by AY(i).

[0342] In addition, the encoding unit 4801 generates an attribute parameter set (APS(i)) corresponding to each frame. In addition, the attribute parameter set of attribute X is represented by AXPS(i), and the attribute parameter set of attribute Y is represented by AYPS(i). The attribute parameter set includes parameters that can be used in decoding the encoded attribute information. The encoded attribute data depends on the corresponding attribute parameter set.

[0343] In addition, the coded attribute data consisting of a plurality of frames is defined as an attribute sequence (Attribute Sequence). The coding unit 4801 generates an attribute sequence parameter set (Attribute Sequence PS: also referred to as attribute SPS) that stores parameters commonly used in decoding processing of a plurality of frames in the attribute sequence. The attribute sequence depends on the attribute SPS.

[0344] Furthermore, in the first encoding method, the encoded attribute data depends on the encoded position data.

[0345] In addition, Fig.37 , an example is shown in which there are two types of attribute information (attribute X and attribute Y). In the case of two types of attribute information, for example, two encoding units generate respective data and metadata. In addition, for example, an attribute sequence is defined for each type of attribute information, and an attribute SPS is generated for each type of attribute information.

[0346] In addition, Fig.37 In FIG. 4 , an example is shown in which there is one type of position information and two types of attribute information, but the present invention is not limited thereto. The attribute information may be one type or three or more types. In this case, the encoded data can be generated in the same way. In addition, in the case of point cloud data without attribute information, there may be no attribute information. In this case, the encoding unit 4801 may not generate a parameter set associated with the attribute information.

[0347] Next, the generation process of the additional information (metadata) is described. The coding unit 4801 generates a parameter set of the entire PCC stream, namely, a PCC stream PS (PCC Stream PS: also referred to as stream PS). The coding unit 4801 stores parameters that can be used in common in the decoding process of one or more position sequences and one or more attribute sequences in the stream PS. For example, the stream PS contains identification information of the codec representing the point group data and information representing the algorithm used in the encoding. The position sequence and the attribute sequence depend on the stream PS.

[0348] Next, the access unit and GOF are described. In this embodiment, the concept of access unit (AccessUnit: AU) and GOF (Group of Frame) is newly introduced.

[0349] An access unit is a basic unit used to access data during decoding, and is composed of one or more data and one or more metadata. For example, an access unit is composed of location information and one or more attribute information at the same time. A GOF is a random access unit, and is composed of one or more access units.

[0350] The coding unit 4801 generates an access unit header (AU Header) as identification information indicating the beginning of the access unit. The coding unit 4801 stores parameters related to the access unit in the access unit header. For example, the access unit header includes: the structure or information of the encoded data included in the access unit. In addition, the access unit header includes parameters commonly used for the data included in the access unit, such as parameters related to the decoding of the encoded data.

[0351] In addition, the encoder 4801 may generate an access unit delimiter that does not include parameters related to the access unit instead of the access unit header. The access unit delimiter is used as identification information indicating the beginning of the access unit. The decoding device recognizes the beginning of the access unit by detecting the access unit header or the access unit delimiter.

[0352] Next, the generation of identification information at the beginning of GOF is described. The coding unit 4801 generates a GOF header (GOFHeader) as identification information indicating the beginning of GOF. The coding unit 4801 stores parameters related to GOF in the GOF header. For example, the GOF header includes: the structure or information of the encoded data included in the GOF. In addition, the GOF header includes parameters commonly used for the data included in the GOF, such as parameters related to the decoding of the encoded data.

[0353] In addition, the encoder 4801 may generate a GOF delimiter that does not include parameters related to GOF instead of the GOF header. The GOF delimiter is used as identification information indicating the beginning of GOF. The decoding device recognizes the beginning of GOF by detecting the GOF header or the GOF delimiter.

[0354] In PCC coded data, for example, an access unit is defined as a PCC frame unit. The decoding device accesses the PCC frame based on identification information at the head of the access unit.

[0355] In addition, for example, GOF is defined as one random access unit. The decoding device accesses the random access unit based on the identification information at the beginning of GOF. For example, if PCC frames have no dependency on each other and can be decoded independently, the PCC frame can also be defined as a random access unit.

[0356] Furthermore, two or more PCC frames may be allocated to one access unit, and a plurality of random access units may be allocated to one GOF.

[0357] Furthermore, the encoder 4801 may define and generate parameter sets or metadata other than those described above. For example, the encoder 4801 may generate SEI (Supplemental Enhancement Information) storing parameters (optional parameters) that may not necessarily be used during decoding.

[0358] Next, the structure of the coded data and the method of storing the coded data in the NAL unit are described.

[0359] For example, a data format is specified for each type of coded data. Fig.38 This is a diagram showing examples of coded data and NAL units.

[0360] For example, Fig.38 As shown, the coded data includes a header and a payload. In addition, the coded data may also include coded data, a header, or length information indicating the length (data amount) of the payload. In addition, the coded data may not include a header.

[0361] The header includes, for example, identification information for identifying data. The identification information indicates, for example, the data type or the frame number.

[0362] The header includes, for example, identification information indicating a reference relationship. The identification information is, for example, information that is saved in the header when there is a dependency relationship between data and is used to refer to the reference target from the reference source. For example, the header of the reference target includes identification information for identifying the data. The header of the reference source includes identification information indicating the reference target.

[0363] Furthermore, when the reference destination or the reference source can be identified or derived from other information, identification information for specifying data or identification information indicating a reference relationship may be omitted.

[0364] The multiplexing unit 4802 stores the coded data in the payload of the NAL unit. The NAL unit header includes pcc_nal_unit_type, which is identification information of the coded data. Fig.39 This is a diagram showing a semantic example of pcc_nal_unit_type.

[0365] like Fig.39 As shown, when pcc_codec_type is codec 1 (Codec 1: first coding method), pcc_nal_unit_type values ​​0 to 10 are allocated to the coded position data (Geometry), coded attribute X data (AttributeX), coded attribute Y data (AttributeY), position PS (Geom.PS), attribute XPS (AttrX.PS), attribute YPS (AttrX.PS), position SPS (Geometry Sequence PS), attribute XSPS (AttributeX Sequence PS), attribute YSPS (AttributeY Sequence PS), AU Header, and GOF Header in codec 1. In addition, values ​​11 and later are allocated as a reserve for codec 1.

[0366] When pcc_codec_type is codec2 (Codec2: second encoding method), pcc_nal_unit_type values ​​0 to 2 are assigned to codec data A (DataA), metadata A (MetaDataA), and metadata B (MetaDataB). In addition, values ​​3 and later are assigned as a reserve for codec2.

[0367] Next, the order in which data is transmitted is described. Next, the constraints on the order in which NAL units are transmitted are described.

[0368] The multiplexing unit 4802 sends out the NAL units in GOF or AU units. The multiplexing unit 4802 arranges the GOF header at the beginning of the GOF and arranges the AU header at the beginning of the AU.

[0369] The multiplexing unit 4802 may configure a sequence parameter set (SPS) for each AU so that even if data is lost due to packet loss or the like, the decoding device can perform decoding from the next AU.

[0370] When there is a dependency relationship related to decoding in the coded data, the decoding device decodes the reference source data after decoding the reference target data. In the decoding device, in order to enable decoding in the order received without rearranging the data, the multiplexing unit 4802 sends the reference target data first.

[0371] Fig.40 This is a diagram showing an example of the order in which NAL units are sent. Fig.40 It shows three examples: position information priority, parameter priority and data merging.

[0372] The sending order with priority on position information is an example in which information related to position information and information related to attribute information are sent together. In the case of this sending order, the sending of information related to position information is completed earlier than the sending of information related to attribute information.

[0373] For example, by using this transmission order, a decoding device that does not decode attribute information may be able to set a time when no processing is performed by ignoring the decoding of attribute information. In addition, for example, in the case of a decoding device that wants to decode position information early, it may be possible to decode the position information earlier by obtaining the encoded data of the position information earlier.

[0374] In addition, Fig.40 In the example, the attribute XSPS and the attribute YSPS are combined and recorded as the attribute SPS, but the attribute XSPS and the attribute YSPS may be configured separately.

[0375] In the parameter set priority sending order, the parameter set is sent first, and then the data is sent.

[0376] As described above, as long as the constraints of the NAL unit sending order are followed, the multiplexing unit 4802 can send the NAL units in any order. For example, the order identification information can also be defined, and the multiplexing unit 4802 has the function of sending NAL units in a plurality of styles. For example, the order identification information of the NAL unit is stored in the stream PS.

[0377] The 3D data decoding device may also perform decoding based on the sequence identification information. The 3D data decoding device may also indicate a desired transmission sequence to the 3D data encoding device, and the 3D data encoding device (multiplexing unit 4802) controls the transmission sequence according to the indicated transmission sequence.

[0378] In addition, as long as the order of sending data is within the range of the constraints of the sending order, such as the sending order of data merging, the multiplexing unit 4802 may generate coded data that combines multiple functions. Fig.40As shown, the GOF header and the AU header may be combined, or the AXPS and the AYPS may be combined. In this case, an identifier indicating that the data has multiple functions is defined in pcc_nal_unit_type.

[0379] The following describes a modified example of the present embodiment. PS has levels, such as frame-level PS, sequence-level PS, and PCC sequence-level PS. If the PCC sequence level is set as the upper level and the frame level is set as the lower level, the parameter storage method can also use the following method.

[0380] The default PS value is represented by the higher-order PS. In addition, when the value of the lower-order PS is different from the value of the upper-order PS, the value of the PS is represented by the lower-order PS. Alternatively, the value of the PS is not recorded in the upper-order PS, and the value of the PS is recorded in the lower-order PS. Alternatively, one or both of the lower-order PS and the upper-order PS are used to represent the information of whether the value of the PS is represented by the lower-order PS, the upper-order PS, or both. Alternatively, the lower-order PS may be merged into the upper-order PS. Alternatively, when the lower-order PS and the upper-order PS are repeated, the multiplexing unit 4802 may omit the sending of one of them.

[0381] In addition, the encoding unit 4801 or the multiplexing unit 4802 may divide the data into slices or tiles, etc., and send the divided data. The divided data includes information for identifying the divided data, and the parameter set includes parameters used in decoding the divided data. In this case, in pcc_nal_unit_type, an identifier indicating that it is data related to a tile or a slice or data storing parameters is defined.

[0382] Next, the processing of the sequence identification information will be described. Fig.41 This is a flowchart of processing performed by the three-dimensional data encoding device (encoding unit 4801 and multiplexing unit 4802) regarding the order in which NAL units are sent.

[0383] First, the 3D data encoding device determines the order of NAL unit transmission (position information priority or parameter set priority) (S4801). For example, the 3D data encoding device determines the transmission order based on a designation from a user or an external device (eg, a 3D data decoding device).

[0384] When the determined sending order is position information priority (position information priority in S4802), the three-dimensional data encoding device sets the order identification information included in the stream PS to position information priority (S4803). That is, in this case, the order identification information indicates that the NAL units are sent in the order of position information priority. And the three-dimensional data encoding device sends the NAL units in the order of position information priority (S4804).

[0385] On the other hand, when the determined transmission order is parameter set priority (parameter set priority in S4802), the three-dimensional data encoding device sets the sequence identification information included in the stream PS to parameter set priority (S4805). That is, in this case, the sequence identification information indicates that the NAL units are transmitted in the order of parameter set priority. And the three-dimensional data encoding device transmits the NAL units in the order of parameter set priority (S4806).

[0386] Fig.42 48 is a flowchart of a process performed by the three-dimensional data decoding device regarding the order in which NAL units are sent. First, the three-dimensional data decoding device analyzes the order identification information included in the stream PS (S4811).

[0387] When the sending order indicated by the sequence identification information is position information-first (position information-first in S4812), the three-dimensional data decoding apparatus decodes the NAL units in the position information-first sending order (S4813).

[0388] On the other hand, when the sending order indicated by the sequence identification information is parameter set priority (parameter set priority in S4812), the three-dimensional data decoding device sets the sending order of the NAL unit to parameter set priority and decodes the NAL unit (S4814).

[0389] For example, when the three-dimensional data decoding apparatus does not decode the attribute information, in step S4813 , it may obtain the NAL unit related to the position information instead of all the NAL units, and decode the position information from the obtained NAL unit.

[0390] Next, the processing related to the generation of AU and GOF is described. Fig.43 This is a flowchart showing the processing performed by the three-dimensional data encoding device (multiplexing unit 4802) regarding the generation of AUs and GOFs in the multiplexing of NAL units.

[0391] First, the three-dimensional data encoding device determines the type of encoded data (S4821). Specifically, the three-dimensional data encoding device determines whether the encoded data to be processed is data starting with AU, data starting with GOF, or other data.

[0392] When the coded data is data starting with GOF (starting with GOF in S4822), the three-dimensional data coding apparatus arranges the GOF header and the AU header at the beginning of the coded data belonging to GOF to generate a NAL unit (S4823).

[0393] When the coded data is data starting with an AU (starting with an AU in S4822), the three-dimensional data coding apparatus arranges the AU header at the beginning of the coded data belonging to the AU and generates a NAL unit (S4824).

[0394] When the coded data does not start with either GOF or AU (other than GOF or AU in S4822), the three-dimensional data coding apparatus arranges the coded data after the AU header of the AU to which the coded data belongs and generates a NAL unit (S4825).

[0395] Next, the processing related to access to AU and GOF is described. Fig.44 This is a flowchart of the processing of the three-dimensional data decoding device related to access to AU and GOF in the inverse multiplexing of NAL units.

[0396] First, the 3D data decoding device determines the type of coded data contained in the NAL unit by parsing nal_unit_type contained in the NAL unit (S4831). Specifically, the 3D data decoding device determines whether the coded data contained in the NAL unit is data starting with AU, data starting with GOF, or other data.

[0397] When the coded data included in the NAL unit is data starting with GOF (starting with GOF in S4832), the three-dimensional data decoding device determines that the NAL unit is the start position of random access, accesses the NAL unit, and starts decoding processing (S4833).

[0398] On the other hand, when the encoded data included in the NAL unit is data starting with an AU (starting with an AU in S4832), the three-dimensional data decoding device determines that the NAL unit starts with an AU, accesses the data included in the NAL unit, and decodes the AU (S4834).

[0399] On the other hand, when the coded data included in the NAL unit is not at the beginning of either GOF or AU (other than the beginning of GOF or AU in S4832), the three-dimensional data decoding apparatus does not process the NAL unit.

[0400] As described above, the three-dimensional data encoding device performs Fig.45The three-dimensional data encoding device encodes time-series three-dimensional data (for example, point group data of a dynamic object). The three-dimensional data includes position information and attribute information at each time.

[0401] First, the three-dimensional data encoding device encodes the position information (S4841). Next, the three-dimensional data encoding device encodes the attribute information of the processing object with reference to the position information at the same time as the attribute information of the processing object (S4842). Fig.37 As shown, the position information and attribute information at the same time constitute an access unit (AU). That is, the three-dimensional data encoding device encodes the attribute information of the processing object with reference to the position information included in the same access unit as the attribute information of the processing object.

[0402] Thus, the three-dimensional data encoding device can use the access unit to facilitate the control of references in encoding. Therefore, the three-dimensional data encoding device can reduce the amount of processing in the encoding process.

[0403] For example, the three-dimensional data encoding device generates a bit stream including information on encoded position information (encoded position data), encoded attribute information (encoded attribute data), and position information indicating a reference destination of the attribute information of a processing target.

[0404] For example, the bitstream includes a position parameter set (position PS) including control information of position information at each time point, and an attribute parameter set (attribute PS) including control information of attribute information at each time point.

[0405] For example, the bitstream includes a position sequence parameter set (position SPS) including common control information in position information at multiple times, and an attribute sequence parameter set (attribute SPS) including common control information in attribute information at multiple times.

[0406] For example, the bitstream includes a stream parameter set (stream PS) including position information at multiple times and control information common to attribute information at multiple times.

[0407] For example, the bitstream includes an access unit header (AU header) including common control information in the access unit.

[0408] For example, the three-dimensional data encoding device encodes GOF (Group of Frames) composed of one or more access units so that they can be decoded independently. That is, GOF is a random access unit.

[0409] For example, the bitstream contains a GOF header containing common control information within the GOF.

[0410] For example, the three-dimensional data encoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0411] In addition, as described above, the three-dimensional data decoding device performs Fig.46 The processing shown. The three-dimensional data decoding device decodes the time-series three-dimensional data (for example, point group data of a dynamic object). The three-dimensional data includes position information and attribute information at each time. The position information and attribute information at the same time constitute an access unit (AU).

[0412] First, the three-dimensional data decoding device decodes the position information from the bit stream (S4851). That is, the three-dimensional data decoding device generates the position information by decoding the encoded position information (encoded position data) included in the bit stream.

[0413] Next, the three-dimensional data decoding device decodes the attribute information of the processing object from the bit stream by referring to the position information at the same time as the attribute information of the processing object (S4852). That is, the three-dimensional data decoding device generates the attribute information by decoding the encoded attribute information (encoded attribute data) contained in the bit stream. At this time, the three-dimensional data decoding device refers to the decoded position information contained in the same access unit as the attribute information.

[0414] Thus, the three-dimensional data decoding device can use the access unit to facilitate the control of references in decoding. Therefore, the three-dimensional data decoding method can reduce the processing amount of the decoding process.

[0415] For example, the three-dimensional data decoding apparatus obtains information indicating the position information of the reference target of the attribute information of the processing target from the bit stream, and decodes the attribute information of the processing target by referring to the position information of the reference target indicated by the obtained information.

[0416] For example, the bitstream includes a position parameter set (position PS) including control information of position information at each time point, and an attribute parameter set (attribute PS) including control information of attribute information at each time point. That is, the 3D data decoding device decodes the position information at the processing target time point using the control information included in the position parameter set at the processing target time point, and decodes the attribute information at the processing target time point using the control information included in the attribute parameter set at the processing target time point.

[0417] For example, the bitstream includes: a position sequence parameter set (position SPS) including common control information in position information at multiple times, and an attribute sequence parameter set (attribute SPS) including common control information in attribute information at multiple times. That is, the three-dimensional data decoding device decodes the position information at multiple times using the control information included in the position sequence parameter set, and decodes the attribute information at multiple times using the control information included in the attribute sequence parameter set.

[0418] For example, the bitstream includes a stream parameter set (stream PS) including control information common to position information at multiple times and attribute information at multiple times. That is, the 3D data decoding device decodes the position information at multiple times and attribute information at multiple times using the control information included in the stream parameter set.

[0419] For example, the bitstream includes an access unit header (AU header) including common control information in the access unit. That is, the 3D data decoding device decodes the position information and attribute information included in the access unit using the control information included in the access unit header.

[0420] For example, the three-dimensional data decoding device independently decodes GOF (Group of Frames) composed of one or more access units. That is, GOF is a random access unit.

[0421] For example, the bitstream includes a GOF header including common control information in the GOF. That is, the three-dimensional data decoding device decodes the position information and attribute information included in the GOF using the control information included in the GOF header.

[0422] For example, the three-dimensional data decoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0423] (Implementation method 5)

[0424] The following describes a method for segmenting point cloud data. Fig.47 The diagram shows an example of slice and tile division.

[0425] First, the method of slice segmentation is described. The three-dimensional data encoding device divides the three-dimensional point group data into arbitrary point groups in slice units. The three-dimensional data encoding device does not divide the position information and attribute information of the constituent points in the slice segmentation, but divides the position information and the attribute information together. That is, the three-dimensional data encoding device performs slice segmentation in a manner that the position information and the attribute information at any point belong to the same slice. In addition, as long as these are followed, any method of segmentation number and segmentation method can be used. In addition, the minimum unit of segmentation is a point. For example, the number of segmentations of the position information and the attribute information is the same. For example, the three-dimensional point corresponding to the position information after the slice segmentation and the three-dimensional point corresponding to the attribute information are included in the same slice.

[0426] In addition, the three-dimensional data encoding device generates slice additional information as additional information related to the number of segments and the segmentation method when the slice is segmented. The slice additional information is the same in the position information and the attribute information. For example, the slice additional information includes information indicating the reference coordinate position, size or side length of the bounding box after segmentation. In addition, the slice additional information includes information indicating the number of segments and the segmentation type, etc.

[0427] Next, a method of tile division is described. The three-dimensional data encoding device divides the slice-divided data into slice position information (G slice) and slice attribute information (A slice), and divides the slice position information and the slice attribute information into tile units.

[0428] In addition, Fig.47 An example of division using an octree structure is shown in the figure, but any number of divisions and any division method may be used.

[0429] In addition, the 3D data encoding device can divide the position information and the attribute information by different division methods or by the same division method. In addition, the 3D data encoding device can divide the multiple slices into tiles by different division methods or by the same division method.

[0430] In addition, the three-dimensional data encoding device generates tile additional information related to the number of divisions and the division method when the tile is divided. The tile additional information (position tile additional information and attribute tile additional information) is independent of the position information and the attribute information. For example, the tile additional information includes information indicating the reference coordinate position, size or side length of the bounding box after division. In addition, the tile additional information includes information indicating the number of divisions and the type of division.

[0431] Next, an example of a method of dividing point cloud data into slices or tiles is described. As a method of dividing into slices or tiles, the three-dimensional data encoding device may use a preset method or adaptively switch the method to be used according to the point cloud data.

[0432] When slicing, the three-dimensional data encoding device divides the three-dimensional space for both the position information and the attribute information. For example, the three-dimensional data encoding device determines the shape of the object and divides the three-dimensional space into slices according to the shape of the object. For example, the three-dimensional data encoding device extracts objects such as trees or buildings and divides them in object units. For example, the three-dimensional data encoding device performs slice segmentation so that the entirety of one or more objects is included in one slice. Alternatively, the three-dimensional data encoding device divides one object into multiple slices.

[0433] In this case, the encoding device may also change the encoding method for each slice, for example. For example, the encoding device may also use a high-quality compression method for a specific object or a specific part of an object. In this case, the encoding device may also save information indicating the encoding method for each slice in additional information (metadata).

[0434] In addition, the three-dimensional data encoding device may also perform slice division based on map information or position information so that each slice corresponds to a preset coordinate space.

[0435] When the tiles are segmented, the three-dimensional data encoding device segments the position information and the attribute information independently. For example, the three-dimensional data encoding device segments the slices into tiles according to the data volume or the processing volume. For example, the three-dimensional data encoding device determines whether the data volume of the slice (for example, the number of three-dimensional points contained in the slice) is greater than a preset threshold. The three-dimensional data encoding device segments the slice into tiles when the data volume of the slice is greater than the threshold. The three-dimensional data encoding device does not segment the slice into tiles when the data volume of the slice is less than the threshold.

[0436] For example, the 3D data encoding device divides the slice into tiles so that the processing amount or processing time in the decoding device is within a certain range (below a preset value). Thus, the processing amount of each tile of the decoding device becomes constant, and distributed processing in the decoding device becomes easy.

[0437] Furthermore, when the processing amounts of position information and attribute information are different, for example, when the processing amount of position information is greater than that of attribute information, the three-dimensional data encoding device divides the position information into a larger number of parts than the attribute information.

[0438] In addition, for example, depending on the content, when the decoding device can decode and display the position information earlier and decode and display the attribute information later, the three-dimensional data encoding device can also divide the position information into more parts than the attribute information. As a result, the decoding device can increase the number of parallel operations of the position information, so the processing of the position information can be faster than the processing of the attribute information.

[0439] Furthermore, the decoding device does not necessarily need to process the sliced ​​or tiled data in parallel, and may determine whether to process them in parallel based on the number or capabilities of the decoding processing units.

[0440] By dividing the data using the above method, adaptive encoding corresponding to the content or object can be realized. In addition, parallel processing in the decoding process can be realized. As a result, the flexibility of the point group encoding system or the point group decoding system is improved.

[0441] Fig.48 This is a diagram showing an example of the style of segmentation of slices and tiles. The DU in the figure is a data unit (DataUnit), which represents the data of a tile or slice. In addition, each DU includes a slice index (Slice Index) and a tile index (TileIndex). The value on the upper right of the DU in the figure represents the slice index, and the value on the lower left of the DU represents the tile index.

[0442] In pattern 1, in slice partitioning, the number of partitions and the partitioning method are the same in G slices and A slices. In tile partitioning, the number of partitions and the partitioning method for G slices are different from the number of partitions and the partitioning method for A slices. In addition, the same number of partitions and the partitioning method are used among multiple G slices. The same number of partitions and the partitioning method are used among multiple A slices.

[0443] In pattern 2, in slice partitioning, the number of partitions and the partitioning method are the same in G slices and A slices. In tile partitioning, the number of partitions and the partitioning method for G slices are different from those for A slices. In addition, the number of partitions and the partitioning method are different between multiple G slices. The number of partitions and the partitioning method are different between multiple A slices.

[0444] (Implementation 6)

[0445] The following is an example of slicing after tile segmentation. In autonomous applications such as automatic driving of vehicles, point group data of the entire area is not required, but point group data of the surrounding area of ​​the vehicle or the area in the direction of travel of the vehicle is required. Here, tiles and slices can be used to selectively decode the original point group data. By dividing the three-dimensional point group data into tiles and then into slices, it is possible to achieve improved coding efficiency or parallel processing. When the data is divided, additional information (metadata) is generated, and the generated additional information is sent to the multiplexing unit.

[0446] Fig.49 This is a block diagram showing the structure of the first coding unit 5010 included in the three-dimensional data coding device related to the present embodiment. The first coding unit 5010 generates coded data (coded stream) by coding the point cloud data using the first coding method (GPCC (Geometry based PCC)). The first coding unit 5010 includes a segmentation unit 5011, a plurality of position information coding units 5012, a plurality of attribute information coding units 5013, an additional information coding unit 5014, and a multiplexing unit 5015.

[0447] The segmentation unit 5011 generates a plurality of segmentation data by segmenting the point group data. Specifically, the segmentation unit 5011 generates a plurality of segmentation data by segmenting the space of the point group data into a plurality of subspaces. Here, the subspace refers to one of a tile and a slice or a combination of a tile and a slice. More specifically, the point group data includes position information, attribute information and additional information. The segmentation unit 5011 segments the position information into a plurality of segmentation position information, and segments the attribute information into a plurality of segmentation attribute information. In addition, the segmentation unit 5011 generates additional information about the segmentation.

[0448] For example, the segmentation unit 5011 first segments the point group into tiles, and then the segmentation unit 5011 further segments the obtained tiles into slices.

[0449] The plurality of position information encoding units 5012 encode the plurality of divided position information to generate a plurality of encoded position information. For example, the plurality of position information encoding units 5012 processes the plurality of divided position information in parallel.

[0450] The multiple attribute information encoding unit 5013 generates multiple coded attribute information by encoding multiple pieces of split attribute information. For example, the multiple attribute information encoding unit 5013 processes multiple pieces of split attribute information in parallel.

[0451] The additional information encoding unit 5014 generates encoded additional information by encoding the additional information included in the point cloud data and the additional information on data division generated by the division unit 5011 during division.

[0452] The multiplexing unit 5015 generates coded data (coded stream) by multiplexing a plurality of coding position information, a plurality of coding attribute information, and coded additional information, and sends out the generated coded data. The coded additional information is used at the time of decoding.

[0453] In addition, Fig.49 , an example is shown in which the number of the position information coding unit 5012 and the number of the attribute information coding unit 5013 are two, but the number of the position information coding unit 5012 and the number of the attribute information coding unit 5013 may be one, or may be more than three. In addition, the plurality of divided data may be processed in parallel in the same chip like the plurality of cores in a CPU, may be processed in parallel by the cores of a plurality of chips, or may be processed in parallel by a plurality of cores in a plurality of chips.

[0454] Next, the decoding process will be described. Fig.50 1 is a block diagram showing the configuration of the first decoding unit 5020. The first decoding unit 5020 restores the point group data by decoding the coded data (coded stream) generated by coding the point group data using the first coding method (GPCC). The first decoding unit 5020 includes a demultiplexing unit 5021, a plurality of position information decoding units 5022, a plurality of attribute information decoding units 5023, an additional information decoding unit 5024, and a combining unit 5025.

[0455] The demultiplexing unit 5021 generates a plurality of encoding position information, a plurality of encoding attribute information, and encoding additional information by demultiplexing the encoded data (encoded stream).

[0456] The plurality of position information decoding units 5022 decode the plurality of coded position information to generate a plurality of divided position information. For example, the plurality of position information decoding units 5022 processes the plurality of coded position information in parallel.

[0457] The multiple attribute information decoding unit 5023 generates multiple pieces of segmented attribute information by decoding multiple pieces of coded attribute information. For example, the multiple attribute information decoding unit 5023 processes multiple pieces of coded attribute information in parallel.

[0458] The plurality of additional information decoding units 5024 generate additional information by decoding the encoded additional information.

[0459] The combining unit 5025 combines the plurality of segmentation position information by using the additional information to generate the position information. The combining unit 5025 combines the plurality of segmentation attribute information by using the additional information to generate the attribute information. For example, the combining unit 5025 first combines the decoded point group data for the slice by using the slice additional information to generate the point group data corresponding to the tile. Then, the combining unit 5025 combines the point group data corresponding to the tile by using the tile additional information to restore the original point group data.

[0460] In addition, Fig.49 , an example is shown in which the number of the position information decoding unit 5022 and the number of the attribute information decoding unit 5023 are two, but the number of the position information decoding unit 5022 and the number of the attribute information decoding unit 5023 may be one, or may be more than three. In addition, the plurality of split data may be processed in parallel in the same chip like the plurality of cores in a CPU, may be processed in parallel by the cores of a plurality of chips, or may be processed in parallel by the plurality of cores in a plurality of chips.

[0461] Next, a method of dividing point cloud data is described. In autonomous applications such as automatic driving of a vehicle, point cloud data is not required for the entire region, but for the area surrounding the vehicle or the region in the vehicle's traveling direction.

[0462] Fig.51 is a diagram showing an example of the shape of a tile. Fig.51 As shown, various shapes such as circle, rectangle, or ellipse can be used as the shape of the tile.

[0463] Fig.52 2 is a diagram showing examples of tiles and slices. The composition of slices may also be different between tiles. For example, the composition of tiles or slices may be optimized based on the amount of data. Alternatively, the composition of tiles or slices may be optimized based on the decoding speed.

[0464] Alternatively, tile division may be performed based on position information. In this case, the attribute information is divided in the same manner as the corresponding position information.

[0465] In addition, in the slicing segmentation after tile segmentation, the location information and attribute information may be segmented into slices by different methods. For example, the slicing segmentation method of each tile may be selected according to a request from an application. Different slicing segmentation methods or tile segmentation methods may also be used based on a request from an application.

[0466] For example, the dividing unit 5011 divides the point cloud data into one or more tiles based on position information such as map information in a two-dimensional shape viewed from above on the three-dimensional point cloud data. Then, the dividing unit 5011 divides each tile into one or more slices.

[0467] In addition, the segmentation unit 5011 may also segment the position information (Geometry) and the attribute information (Attribute) into slices using the same method.

[0468] In addition, the position information and the attribute information may be one type or two or more types, respectively. In addition, in the case of point cloud data without attribute information, there may be no attribute information.

[0469] Fig.53 : is a block diagram of the dividing unit 5011. The dividing unit 5011 includes a tile dividing unit 5031 (Tile Divider), a position information slice dividing unit 5032 (Geometry Slice Divider), and an attribute information slice dividing unit 5033 (Attribute Slice Divider).

[0470] The tile segmentation unit 5031 generates a plurality of tile position information by segmenting the position information (Position (Geometry)) into tiles. In addition, the tile segmentation unit 5031 generates a plurality of tile attribute information by segmenting the attribute information (Attribute) into tiles. In addition, the tile segmentation unit 5031 outputs tile additional information (TileMetaData) including information on tile segmentation and information generated during tile segmentation.

[0471] The position information slice segmentation unit 5032 generates a plurality of segmentation position information (a plurality of slice position information) by segmenting a plurality of tile position information into slices. In addition, the position information slice segmentation unit 5032 outputs position slice additional information (GeometrySlice MetaData) including information on the segmentation of the position information and information generated in the segmentation of the position information.

[0472] The attribute information slice segmentation unit 5033 generates multiple segmentation attribute information (multiple slice attribute information) by segmenting multiple tile attribute information into slices. In addition, the attribute information slice segmentation unit 5033 outputs attribute slice additional information (AttributeSlice MetaData) including information about the slice segmentation of the attribute information and information generated in the slice segmentation of the attribute information.

[0473] Next, an example of the shape of a tile is described. The entire three-dimensional map (3D map) is divided into a plurality of tiles. The data of the plurality of tiles are selectively sent to the three-dimensional data decoding device. Alternatively, the data of the plurality of tiles may be sent to the three-dimensional data decoding device in order from the data of the plurality of tiles with higher importance. Depending on the situation, the shape of the tile may also be selected from a plurality of shapes.

[0474] Fig.54 This is a diagram showing an example of a map in which point cloud data obtained by LiDAR is viewed from above. Fig.54 The example shown is point group data of a highway, including a flyover.

[0475] Fig.55 It means that Fig.54 The figure shows an example of dividing the point cloud data into square tiles. Such square division can be easily performed in the map server. In addition, the height of the tile is set lower than that of a normal road. At the three-dimensional intersection, the height of the tile is set higher than that of a normal road so that the tile includes the three-dimensional intersection.

[0476] Fig.56 It means that Fig.54 The figure shows an example of dividing the point cloud data into circular tiles. In this case, there are cases where adjacent tiles overlap in a top view. When a vehicle needs point cloud data of a surrounding area, the three-dimensional data encoding device sends point cloud data of a cylindrical (circular in a top view) area around the vehicle to the vehicle.

[0477] In addition, Fig.55 Similarly, for a normal road, the height of the tile is set to be lower. At a three-dimensional intersection, the height of the tile is set to be higher than that of a normal road so that the tile includes the three-dimensional intersection.

[0478] The three-dimensional data encoding device may also change the height of the tile, for example, according to the shape or height of a road or a building. In addition, the three-dimensional data encoding device may also change the height of the tile according to location information or area information. In addition, the three-dimensional data encoding device may also change the height of the tile for each tile. Alternatively, the three-dimensional data encoding device may also change the height of the tile for each interval including a plurality of tiles. That is, the three-dimensional data encoding device may also set the heights of a plurality of tiles within an interval to be the same. In addition, tiles of different heights may also be repeated in a bird's eye view.

[0479] Fig.57 This is a diagram showing an example of tile division when using tiles of various shapes, sizes, or heights. The tile may be of any shape, any size, or a combination thereof.

[0480] For example, the three-dimensional data encoding device may be divided into square tiles without repetition or circular tiles without repetition. In addition, the shape of the tile may not be square or circular, but may be a polygon with more than three vertices or a shape without vertices.

[0481] In addition, the shape of the tile can be more than two, or tiles of different shapes can be repeated. In addition, the type of the shape of the tile can be more than one, and in the same shape of the segmentation, shapes of different sizes are combined, and they can also be repeated.

[0482] For example, in a region without an object such as a road, a tile larger than a region with an object is used. In addition, the three-dimensional data encoding device may adaptively change the shape or size of the tile according to the object.

[0483] In addition, for example, since the three-dimensional data encoding device is more likely to need to read tiles far in front of the car as the direction of travel of the car (vehicle), the tiles in the direction of travel can be set to a larger size, and since the possibility of the car moving to the side of the car is lower, the tiles on the side can be set to a smaller size than the tiles in the direction of travel.

[0484] Fig.58 This is a diagram showing an example of tile data stored in a server. For example, point cloud data is tile-divided and encoded in advance, and the obtained encoded data is stored in the server. The user obtains the desired tile data from the server when necessary. Alternatively, the server (three-dimensional data encoding device) can also perform tile division and encoding according to the user's instructions to include the data desired by the user.

[0485] For example, when the moving object (vehicle) moves at a high speed, it is conceivable that a larger range of point cloud data is required. Therefore, the server may also determine the shape and size of the tile and perform tile segmentation based on the pre-estimated vehicle speed (for example, the vehicle speed that can be estimated based on the legal speed of the road, the width and shape of the road, or the statistical speed, etc.). Or, Fig.58 As shown, the server may also encode tiles of multiple shapes or sizes in advance and store the obtained data. The moving body may also obtain the data of tiles of appropriate shapes and sizes according to the moving direction and speed of the moving body.

[0486] Fig.59 is a diagram showing an example of a system for tile partitioning. Fig.59 As shown, the shape and area of ​​the tile may also be determined based on the position of the antenna (base station) as a communication means for transmitting the point group data or the communication area supported by the antenna. Alternatively, when the point group data is generated by a sensor such as a camera, the shape and area of ​​the tile may also be determined based on the position of the sensor or the object range (detection range) of the sensor.

[0487] One tile may be assigned to one antenna or sensor, or one tile may be assigned to multiple antennas or sensors. Multiple tiles may be assigned to one antenna or sensor. Antennas or sensors may be fixed or movable.

[0488] For example, the coded data divided into tiles can also be managed by a server connected to an antenna or sensor corresponding to the area assigned to the tile. The server can also manage the coded data of the area and the tile information of the adjacent area. It is also possible to manage multiple coded data of multiple tiles in a centralized management server (cloud) that manages multiple servers corresponding to each tile. Alternatively, it is also possible not to set up a server corresponding to the tile, and the antenna or sensor is directly connected to the centralized management server.

[0489] In addition, the target range of the antenna or sensor may vary depending on the power of the radio wave, the difference in equipment and the setting conditions, and the shape and size of the tile may also vary to match these. Based on the target range of the antenna or sensor, it is also possible to allocate slices instead of tiles, or to allocate PCC frames.

[0490] Next, a method of dividing a tile into slices is described. By assigning similar objects to the same slice, encoding efficiency can be improved.

[0491] For example, the three-dimensional data encoding device recognizes objects (roads, buildings, trees, etc.) using features of point group data, and performs slice division by clustering the point groups for each object.

[0492] Alternatively, the three-dimensional data encoding device may group objects having the same attributes and perform slice segmentation by assigning slices to each group. Here, the attribute is, for example, information about movement, and the objects are grouped by classifying them into dynamic information such as pedestrians and cars, quasi-dynamic information such as accidents and congestion, quasi-static information such as traffic restrictions and road works, and static information such as road surfaces and structures.

[0493] In addition, data may be repeated in a plurality of slices. For example, when slice division is performed for each of a plurality of object groups, any object may belong to one object group or to two or more object groups.

[0494] Fig.60 is a diagram showing an example of the slice segmentation. Fig.60 In the example shown, the tile is a rectangular parallelepiped, but the tile may also be cylindrical or have other shapes.

[0495] The point groups included in the tiles are grouped into object groups such as roads, buildings, and trees, for example. Then, the tiles are sliced ​​so that each object group is included in one slice. Then, each slice is encoded separately.

[0496] Next, the encoding method of the segmented data is described. The three-dimensional data encoding device (first encoding unit 5010) encodes the segmented data separately. When encoding the attribute information, the three-dimensional data encoding device generates dependency information indicating which composition information (position information, additional information or other attribute information) is encoded as additional information. That is, the dependency information, for example, indicates composition information of a reference target (dependent target). In this case, the three-dimensional data encoding device generates the dependency information based on the composition information corresponding to the segmented shape of the attribute information. In addition, the three-dimensional data encoding device can also generate the dependency information based on the composition information corresponding to multiple segmented shapes.

[0497] The three-dimensional data encoding device may generate the dependency information and send the generated dependency information to the three-dimensional data decoding device. Alternatively, the three-dimensional data decoding device may generate the dependency information and the three-dimensional data encoding device may not send the dependency information. Furthermore, the three-dimensional data encoding device may pre-set the dependency to be used and the three-dimensional data encoding device may not send the dependency information.

[0498] Fig.61 The figure is an example of a dependency relationship between each data. The tip of the arrow in the figure represents the dependency target, and the root of the arrow represents the dependency source. The three-dimensional data decoding device decodes the data in the order from the dependency target to the dependency source. In addition, the data represented by the solid line in the figure is the data actually sent, and the data represented by the dotted line is the data that has not been sent.

[0499] In this figure, G represents position information and A represents attribute information. t1 Indicates the location information of tile number 1, G t2 Indicates the location information of tile number 2. t1s1 Indicates the location information of tile number 1 and slice number 1, G t1s2 Indicates the location information of tile number 1 and slice number 2, G t2s1 Indicates the location information of tile number 2 and slice number 1, G t2s2 Indicates the location information of tile number 2 and slice number 2. Similarly, A t1 Indicates the attribute information of tile number 1, A t2 Indicates the attribute information of tile number 2. t1s1 Indicates the attribute information of tile number 1 and slice number 1, A t1s2 Indicates the attribute information of tile number 1 and slice number 2, A t2s1 Indicates the attribute information of tile number 2 and slice number 1, A t2s2 Indicates the attribute information of tile number 2 and slice number 2.

[0500] Mtile represents tile additional information, MGslice represents position slice additional information, and MAslice represents attribute slice additional information. t1s1 Indicates attribute information A t1s1 Dependency information of D t2s1 Indicates attribute information A t2s1 Dependency information.

[0501] In addition, different tile partitioning or slice partitioning structures may be used depending on the application, etc.

[0502] Furthermore, the 3D data encoding device may rearrange the data in a decoding order so that the 3D data decoding device does not need to rearrange the data. Furthermore, the data may be rearranged in the 3D data decoding device, or the data may be rearranged in both the 3D data encoding device and the 3D data decoding device.

[0503] Fig.62 is a diagram showing an example of the order in which data is decoded. Fig.62 In the example, the decoding is performed sequentially from the data on the left. The three-dimensional data decoding device decodes the data in the dependency relationship from the data of the dependent target first. For example, the three-dimensional data encoding device pre-sorts the data into this order and sends it out. In addition, any order is acceptable as long as the data of the dependent target comes first. In addition, the three-dimensional data encoding device may also send the additional information and the dependency relationship information before the data.

[0504] In addition, the three-dimensional data decoding device may selectively decode tiles based on a request from an application and information obtained from a NAL unit header. Fig.63 This is a diagram showing an example of coded data of tiles. For example, the decoding order of tiles is arbitrary. In other words, there may be no dependency between tiles.

[0505] Next, the configuration of the combining unit 5025 included in the first decoding unit 5020 will be described. Fig.64 2 is a block diagram showing the configuration of the combining unit 5025. The combining unit 5025 includes a location information slice combining unit 5041 (Geometry Slice Combiner), an attribute information slice combining unit 5042 (Attribute Slice Combiner), and a tile combining unit (Tile Combiner).

[0506] The position information slice combining unit 5041 combines a plurality of split position information using position slice additional information to generate a plurality of tile position information. The attribute information slice combining unit 5042 combines a plurality of split attribute information using attribute slice additional information to generate a plurality of tile attribute information.

[0507] The tile combining unit 5043 combines a plurality of tile position information using the tile additional information to generate position information. In addition, the tile combining unit 5043 combines a plurality of tile attribute information using the tile additional information to generate attribute information.

[0508] In addition, the number of divided slices or tiles is 1 or more. That is, the division of slices or tiles may not be performed.

[0509] Next, the structure of the coded data after being divided into slices or tiles and the method of storing the coded data in the NAL unit (multiplexing method) are described. Fig.65 This is a diagram showing the structure of coded data and a method of storing coded data in a NAL unit.

[0510] The encoded data (segment position information and segment attribute information) is stored in the payload of the NAL unit.

[0511] The encoded data includes a header and a payload. The header contains identification information for determining the data contained in the payload. The identification information includes, for example, the type of slice segmentation or tile segmentation (slice_type, tile_type), index information for determining the slice or tile (slice_idx, tile_idx), location information of the data (slice or tile) or the address of the data (address), etc. The index information for determining the slice is also recorded as a slice index (SliceIndex). The index information for determining the tile is also recorded as a tile index (TileIndex). In addition, the type of segmentation is, for example, a method based on object shape as described above, a method based on map information or location information, or a method based on data volume or processing volume, etc.

[0512] In addition, the header of the encoded data contains identification information indicating the dependency relationship. That is, the header contains identification information for referencing the dependent target from the dependent source when there is a dependency relationship between the data. For example, the header of the data of the dependent target contains identification information for determining the data. The header of the data of the dependent source contains identification information indicating the dependent target. In addition, in the case where the identification information for determining the data, additional information about slice segmentation or tile segmentation, and identification information indicating the dependency relationship can be identified or derived based on other information, this information can also be omitted.

[0513] Next, the flow of encoding and decoding processing of point cloud data according to the present embodiment will be described. Fig.66 This is a flowchart of the encoding process of the point cloud data according to the present embodiment.

[0514] First, the three-dimensional data encoding device determines the segmentation method to be used (S5011). The segmentation method includes whether to perform tile segmentation or slice segmentation. In addition, the segmentation method may also include the number of segmentations and the type of segmentation when performing tile segmentation or slice segmentation. The type of segmentation refers to the above-mentioned method based on object shape, the method based on map information or location information, or the method based on data volume or processing volume. In addition, the segmentation method may also be preset.

[0515] When tile segmentation is performed (Yes in S5012), the three-dimensional data encoding device generates a plurality of tile position information and a plurality of tile attribute information by segmenting the position information and the attribute information together (S5013). In addition, the three-dimensional data encoding device generates tile additional information related to tile segmentation. In addition, the three-dimensional data encoding device may also segment the position information and the attribute information independently.

[0516] When slice segmentation is performed (Yes in S5014), the three-dimensional data encoding device generates a plurality of segmentation position information and a plurality of segmentation attribute information (S5015) by independently segmenting a plurality of tile position information and a plurality of tile attribute information (or position information and attribute information). In addition, the three-dimensional data encoding device generates position slice additional information and attribute slice additional information related to the slice segmentation. In addition, the three-dimensional data encoding device may also segment the tile position information and the tile attribute information together.

[0517] Next, the three-dimensional data encoding device generates a plurality of coded position information and a plurality of coded attribute information by encoding each of the plurality of segmentation position information and the plurality of segmentation attribute information (S5016). In addition, the three-dimensional data encoding device generates dependency relationship information.

[0518] Next, the three-dimensional data encoding device generates encoded data (encoded stream) by NAL unitizing (multiplexing) the plurality of encoding position information, the plurality of encoding attribute information and the additional information (S5017). In addition, the three-dimensional data encoding device sends out the generated encoded data.

[0519] Fig.67 Flowchart of decoding processing of point cloud data related to the present embodiment. First, the three-dimensional data decoding device determines the segmentation method (S5021) by parsing the additional information (tile additional information, position slice additional information and attribute slice additional information) related to the segmentation method contained in the coded data (coded stream). The segmentation method includes whether to perform tile segmentation and whether to perform slice segmentation. In addition, the segmentation method may also include the number of segmentations and the type of segmentation when performing tile segmentation or slice segmentation.

[0520] Next, the three-dimensional data decoding apparatus generates segmentation position information and segmentation attribute information by decoding the plurality of pieces of encoding position information and the plurality of pieces of encoding attribute information included in the encoded data using the dependency information included in the encoded data (S5022).

[0521] When the additional information indicates that the slice segmentation has been performed (Yes in S5023), the three-dimensional data decoding device generates a plurality of tile position information and a plurality of tile attribute information by combining the plurality of segmentation position information and the plurality of segmentation attribute information by respective methods based on the position slice additional information and the attribute slice additional information (S5024). Alternatively, the three-dimensional data decoding device may combine the plurality of segmentation position information and the plurality of segmentation attribute information by the same method.

[0522] When the additional information indicates that tile segmentation has been performed (Yes in S5025), the three-dimensional data decoding device generates position information and attribute information by combining multiple tile position information and multiple tile attribute information (multiple segmentation position information and multiple segmentation attribute information) in the same way based on the tile additional information (S5026). Alternatively, the three-dimensional data decoding device may combine multiple tile position information and multiple tile attribute information in different ways.

[0523] Next, the tile additional information will be described. The three-dimensional data encoding device generates tile additional information as metadata regarding a tile division method, and transmits the generated tile additional information to the three-dimensional data decoding device.

[0524] Fig.68 This is a diagram showing an example of the syntax of tile additional information (TileMetaData). Fig.68 As shown, for example, tile additional information includes segmentation method information (type_of_divide), shape information (topview_shape), repeat flag (tile_overlap_flag), repeat information (type_of_overlap), height information (tile_height), number of tiles (tile_number) and tile position information (global_position, relative_position).

[0525] The division method information (type_of_divide) indicates a division method of the tile. For example, the division method information indicates whether the division method of the tile is division based on map information, that is, division based on a bird's-eye view (top_view), or other than this (other).

[0526] The shape information (topview_shape) is included in the tile additional information, for example, when the tile segmentation method is based on the top view segmentation. The shape information indicates the shape of the tile when viewed from above. For example, the shape includes a square and a circle. In addition, the shape may include a polygon other than an ellipse, a rectangle or a quadrilateral, or other shapes. In addition, the shape information is not limited to the shape of the tile when viewed from above, and may also indicate the three-dimensional shape of the tile (such as a cube and a cylinder, etc.).

[0527] The repeat flag (tile_overlap_flag) indicates whether a tile is repeated. For example, when the tile segmentation method is based on the top-view segmentation, the repeat flag is included in the tile additional information. In this case, the repeat flag indicates whether the tile is repeated in the top view. In addition, the repeat flag can also indicate whether the tile is repeated in three-dimensional space.

[0528] The overlap information (type_of_overlap) is included in the tile additional information when the tile overlaps, for example. The overlap information indicates the tile overlap method, etc. For example, the overlap information indicates the size of the overlapped area, etc.

[0529] The height information (tile_height) indicates the height of the tile. In addition, the height information may also include information indicating the shape of the tile. For example, if the top view shape of the tile is a rectangle, the information may also indicate the length of the side of the rectangle (the vertical length and the horizontal length). In addition, if the top view shape of the tile is a circle, the information may also indicate the diameter or radius of the circle.

[0530] In addition, the height information may indicate the height of each tile or the common height of multiple tiles. In addition, multiple height types of roads and three-dimensional intersections may be pre-set, and the height of each height type and the height type of each tile may be indicated by the height information. Alternatively, the height of each height type may be pre-defined, and the height type of each tile may be indicated by the height information. That is, the height of each height type may not be indicated by the height information.

[0531] The tile number (tile_number) indicates the number of tiles. In addition, the tile additional information may include information indicating the interval between tiles.

[0532] The tile position information (global_position, relative_position) is information used to determine the position of each tile. For example, the tile position information indicates the absolute coordinates or relative coordinates of each tile.

[0533] In addition, part or all of the above information may be set for each tile, or may be set for each plurality of tiles (for example, for each frame or for each plurality of frames).

[0534] The three-dimensional data encoding device may include the tile additional information in SEI (Supplemental Enhancement Information) and send it out. Alternatively, the three-dimensional data encoding device may store the tile additional information in an existing parameter set (PPS, GPS, APS, etc.) and send it out.

[0535] For example, when the tile additional information changes for each frame, the tile additional information may be stored in the parameter set (GPS or APS, etc.) of each frame. When the tile additional information does not change within a sequence, the tile additional information may be stored in the parameter set (position SPS or attribute SPS) of each sequence. Furthermore, when the same tile segmentation information is used in the position information and the attribute information, the tile additional information may be stored in the parameter set (stream PS) of the PCC stream.

[0536] In addition, the tile additional information can be stored in any one of the above parameter sets or in multiple parameter sets. In addition, the tile additional information can also be stored in the header of the encoded data. In addition, the tile additional information can also be stored in the header of the NAL unit.

[0537] Furthermore, all or part of the tile additional information may be saved in one of the headers of the split position information and the headers of the split attribute information, but not in the other. For example, in a case where the same tile additional information is used in the position information and the attribute information, the tile additional information may be included in the header of one of the position information and the attribute information. For example, in a case where the attribute information depends on the position information, the position information is processed first. Therefore, the tile additional information may be included in the header of the position information, and the tile additional information may not be included in the header of the attribute information. In this case, the three-dimensional data decoding device, for example, determines that the attribute information of the dependent source belongs to the same tile as the tile of the dependent target position information.

[0538] The three-dimensional data decoding device reconstructs the point group data divided by the tiles based on the tile additional information. When there is repeated point group data, the three-dimensional data decoding device determines the repeated point group data and selects one or merges the multiple point group data.

[0539] In addition, the three-dimensional data decoding device may also use tile additional information for decoding. For example, the three-dimensional data decoding device may also decode each tile when multiple tiles are repeated, and perform processing (such as smoothing or filtering) using the decoded multiple data to generate point group data. This may enable higher-precision decoding.

[0540] Fig.69 3D data encoding device and a 3D data decoding device. The tile segmentation unit 5051 segments the point cloud data including the position information and the attribute information into the first tile and the second tile. In addition, the tile segmentation unit 5051 sends the tile additional information related to the tile segmentation to the decoding unit 5053 and the tile combining unit 5054.

[0541] The encoding unit 5052 generates encoded data by encoding the first tile and the second tile.

[0542] The decoding unit 5053 restores the first tile and the second tile by decoding the coded data generated by the coding unit 5052. The tile combining unit 5054 combines the first tile and the second tile using the tile additional information to restore the point cloud data (position information and attribute information).

[0543] Next, the slice additional information will be described. The three-dimensional data encoding device generates slice additional information as metadata regarding a method of dividing a slice, and transmits the generated slice additional information to the three-dimensional data decoding device.

[0544] Fig.70 This is a diagram showing an example of the syntax of slice additional information (SliceMetaData). Fig.70 As shown, for example, the slice additional information includes division method information (type_of_divide), a repeat flag (slice_overlap_flag), repeat information (type_of_overlap), the number of slices (slice_number), slice position information (global_position, relative_position) and slice size information (slice_bounding_box_size).

[0545] The segmentation method information (type_of_divide) indicates the segmentation method of the slice. For example, the segmentation method information indicates whether the segmentation method of the slice is Fig.60 In the example, the information of the slice additionally includes information indicating the method of object segmentation. For example, the information indicates whether one object is segmented into a plurality of slices or allocated to a single slice. In addition, the information may also indicate the number of segmentations when one object is segmented into a plurality of slices.

[0546] The overlap flag (slice_overlap_flag) indicates whether the slice is overlapped. The overlap information (type_of_overlap) is included in the slice additional information, for example, when the slice is overlapped. The overlap information indicates the overlap method of the slice, etc. For example, the overlap information indicates the size of the overlapped area, etc.

[0547] The slice number (slice_number) indicates the number of slices.

[0548] The slice position information (global_position, relative_position) and the slice size information (slice_bounding_box_size) are information about the area of ​​the slice. The slice position information is information used to determine the position of each slice. For example, the slice position information indicates the absolute coordinates or relative coordinates of each slice. The slice size information (slice_bounding_box_size) indicates the size of each slice. For example, the slice size information indicates the size of the bounding box of each slice.

[0549] The three-dimensional data encoding device may include the slice additional information in the SEI and send it out. Alternatively, the three-dimensional data encoding device may store the slice additional information in an existing parameter set (PPS, GPS, APS, etc.) and send it out.

[0550] For example, when the slice additional information changes for each frame, the slice additional information may be stored in the parameter set (GPS or APS, etc.) of each frame. When the slice additional information does not change within a sequence, the slice additional information may be stored in the parameter set (position SPS or attribute SPS) of each sequence. Furthermore, when the same slice segmentation information is used in the position information and the attribute information, the slice additional information may be stored in the parameter set (stream PS) of the PCC stream.

[0551] In addition, the slice additional information can be stored in any of the above parameter sets or in multiple parameter sets. In addition, the slice additional information can also be stored in the header of the encoded data. In addition, the slice additional information can also be stored in the header of the NAL unit.

[0552] Furthermore, all or part of the slice additional information may be stored in one of the headers of the segmented position information and the headers of the segmented attribute information, but not in the other. For example, in a case where the same slice additional information is used in the position information and the attribute information, the slice additional information may be included in the header of one of the position information and the attribute information. For example, in a case where the attribute information depends on the position information, the position information is processed first. Therefore, the slice additional information may be included in the header of the position information, and not included in the header of the attribute information. In this case, the three-dimensional data decoding device determines, for example, that the attribute information of the dependent source belongs to the same slice as the slice of the position information of the dependent target.

[0553] The 3D data decoding device reconstructs the point group data divided by the slices based on the slice additional information. If there is repeated point group data, the 3D data decoding device determines the repeated point group data and selects one or merges the multiple point group data.

[0554] In addition, the 3D data decoding device may also use slice additional information for decoding. For example, the 3D data decoding device may also decode each slice when multiple slices are repeated, and perform processing (such as smoothing or filtering) using the decoded multiple data to generate point group data. This may enable higher-precision decoding.

[0555] Fig.71 This is a flowchart of a three-dimensional data encoding process including a process for generating tile additional information, which is performed by the three-dimensional data encoding device according to the present embodiment.

[0556] First, the three-dimensional data encoding device determines a method for segmenting tiles (S5031). Specifically, the three-dimensional data encoding device determines whether to use a segmentation method based on a top view (top_view) or a method other than the top view (other) as a segmentation method for tiles. In addition, the three-dimensional data encoding device determines the shape of the tile when the segmentation method based on a top view is used. In addition, the three-dimensional data encoding device determines whether the tile overlaps with other tiles.

[0557] When the tile division method determined in step S5031 is a division method based on a top view (Yes in S5032), the three-dimensional data encoding apparatus records that the tile division method is a division method based on a top view (top_view) in the tile additional information (S5033).

[0558] On the other hand, when the tile segmentation method determined by step S5031 is a method other than the top view segmentation method (No in S5032), the three-dimensional data encoding device records the fact that the tile segmentation method is a method other than the top view segmentation method (top_view) in the tile additional information (S5034).

[0559] In addition, when the shape of the top-view tile determined in step S5031 is a square (square in S5035), the three-dimensional data encoding device records the fact that the shape of the top-view tile is a square in the tile additional information (S5036). On the other hand, when the shape of the top-view tile determined in step S5031 is a circle (circle in S5035), the three-dimensional data encoding device records the fact that the shape of the top-view tile is a circle in the tile additional information (S5037).

[0560] Next, the three-dimensional data encoding device determines whether the tile is repeated with other tiles (S5038). If the tile is repeated with other tiles (Yes in S5038), the three-dimensional data encoding device records the tile duplication in the tile additional information (S5039). On the other hand, if the tile is not repeated with other tiles (No in S5038), the three-dimensional data encoding device records the tile non-duplication in the tile additional information (S5040).

[0561] Next, the three-dimensional data encoding device divides the tile based on the tile division method determined in step S5031, encodes each tile, and sends out the generated encoded data and tile additional information (S5041).

[0562] Fig.72 This is a flowchart of a three-dimensional data decoding process using tile additional information performed by the three-dimensional data decoding device according to the present embodiment.

[0563] First, the three-dimensional data decoding apparatus parses tile additional information included in a bit stream (S5051).

[0564] When the tile additional information indicates that the tile does not overlap with other tiles (No in S5052), the three-dimensional data decoding device generates point cloud data of each tile by decoding each tile (S5053). Next, the three-dimensional data decoding device reconstructs point cloud data from the point cloud data of each tile based on the tile segmentation method and the shape of the tile indicated by the tile additional information (S5054).

[0565] On the other hand, when the tile additional information indicates that the tile is repeated with other tiles (Yes in S5052), the three-dimensional data decoding device generates point group data of each tile by decoding each tile. In addition, the three-dimensional data decoding device determines the repeated part of the tile based on the tile additional information (S5055). In addition, the three-dimensional data decoding device can also use repeated multiple information to perform decoding processing on the repeated part. Next, the three-dimensional data decoding device reconstructs the point group data according to the point group data of each tile based on the tile segmentation method, tile shape and repeated information indicated by the tile additional information (S5056).

[0566] The following describes a modification example of slices. The three-dimensional data encoding device may also send information indicating the type (road, building, tree, etc.) or attribute (dynamic information, static information, etc.) of the object as additional information. Alternatively, the encoding parameters may be predefined according to the object, and the three-dimensional data encoding device notifies the three-dimensional data decoding device of the encoding parameters by sending the type or attribute of the object.

[0567] The following method can also be used for the encoding order and sending order of slice data. For example, the three-dimensional data encoding device can also encode the slice data in sequence starting from the data that is easier to identify or cluster the object. Alternatively, the three-dimensional data encoding device can also encode in sequence starting from the slice data whose clustering ends earlier. In addition, the three-dimensional data encoding device can also send out in sequence starting from the encoded slice data. Alternatively, the three-dimensional data encoding device can also send out the slice data in order from high to low decoding priority in the application. For example, in the case where the decoding priority of dynamic information is higher, the three-dimensional data encoding device can also send out the slice data in sequence starting from the slices grouped by dynamic information.

[0568] Furthermore, when the order of the encoded data is different from the order of the decoding priority, the three-dimensional data encoding device may rearrange the encoded data before sending it out. Furthermore, when accumulating the encoded data, the three-dimensional data encoding device may rearrange the encoded data before accumulating it.

[0569] The application (3D data decoding device) requests the server (3D data encoding device) to send slices containing desired data. The server may send the slice data required by the application and not send unnecessary slice data.

[0570] The application requests the server to send the tile containing the desired data. The server may also send the tile data required by the application and not send the tile data that is not required.

[0571] As described above, the three-dimensional data encoding device according to this embodiment performs Fig.73 First, the three-dimensional data encoding device generates a plurality of encoded data (S5061) by encoding a plurality of subspaces (e.g., tiles) obtained by dividing an object space containing a plurality of three-dimensional points. The three-dimensional data encoding device generates a bit stream (S5062) including the plurality of encoded data and first information (e.g., topview_shape) indicating the shapes of the plurality of subspaces.

[0572] Thus, the three-dimensional data encoding device can select an arbitrary shape from a plurality of types of subspace shapes, and thus can improve encoding efficiency.

[0573] For example, the above-mentioned shape is a two-dimensional shape or a three-dimensional shape of the above-mentioned multiple subspaces. For example, the above-mentioned shape is a shape of the above-mentioned multiple subspaces viewed from above. That is, the first information indicates the shape of the subspace observed from a specific direction (for example, from above). In other words, the first information indicates the shape of the subspace viewed from above. For example, the above-mentioned shape is a rectangle or a circle.

[0574] For example, the bitstream includes second information (eg, tile_overlap_flag) indicating whether the plurality of sub-intervals overlap.

[0575] As a result, the three-dimensional data encoding device can repeat the subspace, and thus can generate the subspace without complicating the shape of the subspace.

[0576] For example, the bitstream includes third information (eg, type_of_divide) indicating whether the division method of the plurality of sub-intervals is a division method using a bird's-eye view.

[0577] For example, the bitstream includes fourth information (eg, tile_height) indicating at least one of the height, width, depth, and radius of the plurality of sub-intervals.

[0578] For example, the bit stream includes fifth information (for example, global_position or relative_position) indicating the position of each of the plurality of sub-intervals.

[0579] For example, the bit stream includes sixth information (eg, tile_number) indicating the number of the plurality of sub-intervals.

[0580] For example, the bit stream includes seventh information indicating intervals between the plurality of subintervals.

[0581] For example, the three-dimensional data encoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0582] In addition, the three-dimensional data decoding device of this embodiment performs Fig.74 The processing shown. First, the three-dimensional data decoding device restores the above-mentioned multiple subspaces (S5071) by decoding multiple encoded data generated by encoding multiple subspaces (such as tiles) divided into an object space containing multiple three-dimensional points, which are contained in the bit stream. The three-dimensional data decoding device restores the above-mentioned object space (S5072) by combining the above-mentioned multiple subspaces using the first information (such as topview_shape) contained in the above-mentioned bit stream and indicating the shapes of the above-mentioned multiple subspaces. For example, the three-dimensional data decoding device can grasp the position and range of each subspace in the object space by identifying the shapes of the multiple subspaces using the first information. The three-dimensional data decoding device can combine the multiple subspaces based on the grasped positions and ranges of the multiple subspaces. Thereby, the three-dimensional data decoding device can correctly combine the multiple subspaces.

[0583] For example, the shape is a two-dimensional shape or a three-dimensional shape of the plurality of subspaces. For example, the shape is a rectangle or a circle.

[0584] For example, the bitstream includes second information (e.g., tile_overlap_flag) indicating whether the plurality of subspaces are repeated. The three-dimensional data decoding device further uses the second information to combine the plurality of subspaces in the restoration of the object space. For example, the three-dimensional data decoding device uses the second information to determine whether the subspace is repeated. If the subspace is repeated, the three-dimensional data decoding device determines the repeated area and performs a predetermined correspondence on the determined repeated area.

[0585] For example, the bitstream includes third information (e.g., type_of_divide) indicating whether the division method of the plurality of sub-intervals is a division method using a bird's-eye view. When the third information indicates that the division method of the plurality of sub-intervals is a division method using a bird's-eye view, the three-dimensional data decoding device combines the plurality of subspaces using the first information.

[0586] For example, the bitstream includes fourth information (e.g., tile_height) indicating at least one of the height, width, depth, and radius of the plurality of sub-intervals. The three-dimensional data decoding device further uses the fourth information to combine the plurality of subspaces in the restoration of the object space. For example, the three-dimensional data decoding device can grasp the position and range of each subspace in the object space by identifying the height of the plurality of subspaces using the fourth information. The three-dimensional data decoding device can combine the plurality of subspaces based on the grasped positions and ranges of the plurality of subspaces.

[0587] For example, the bit stream includes fifth information (e.g., global_position or relative_position) indicating the position of each of the plurality of subspaces. The three-dimensional data decoding device further combines the plurality of subspaces using the fifth information in restoring the object space. For example, the three-dimensional data decoding device can grasp the position of each subspace in the object space by identifying the positions of the plurality of subspaces using the fifth information. The three-dimensional data decoding device can combine the plurality of subspaces based on the grasped positions of the plurality of subspaces.

[0588] For example, the bitstream includes sixth information (eg, tile_number) indicating the number of the plurality of subspaces. The three-dimensional data decoding device further combines the plurality of subspaces using the sixth information in restoring the target space.

[0589] For example, the bit stream includes seventh information indicating the intervals of the plurality of sub-intervals. The three-dimensional data decoding device further combines the plurality of subspaces using the seventh information in restoring the object space. For example, the three-dimensional data decoding device can grasp the position and range of each subspace in the object space by identifying the intervals of the plurality of subspaces using the seventh information. The three-dimensional data decoding device can combine the plurality of subspaces based on the grasped positions and ranges of the plurality of subspaces.

[0590] For example, the three-dimensional data decoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0591] As mentioned above, although the three-dimensional data encoding device and the three-dimensional data decoding device etc. which concern on embodiment of this invention are demonstrated, this invention is not limited to this embodiment.

[0592] In addition, each processing unit included in the three-dimensional data encoding device and the three-dimensional data decoding device according to the above-mentioned embodiment is typically implemented as an integrated circuit, that is, LSI. They may be formed individually as one chip, or part or all of them may be included as one chip.

[0593] In addition, integrated circuits are not limited to LSIs, and can also be implemented by dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays) that can be programmed after LSI manufacturing, or reconfigurable processors that can reconfigure the connections or settings of circuit cells within LSIs can also be used.

[0594] In addition, in each of the above-mentioned embodiments, each component may be formed by dedicated hardware, or implemented by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

[0595] Furthermore, the present invention may be implemented as a three-dimensional data encoding method or a three-dimensional data decoding method executed by a three-dimensional data encoding device or a three-dimensional data decoding device.

[0596] In addition, the division of the functional blocks in the block diagram is an example, and multiple functional blocks can also be implemented as one functional block, or one functional block can be divided into multiple blocks, or a part of the function can be transferred to other functional blocks. In addition, it is also possible that a single hardware or software processes the functions of multiple functional blocks with similar functions in parallel or in a time-sharing manner.

[0597] In addition, the execution order of each step in the flowchart is exemplified for the purpose of specifically describing the present invention, and may be an order other than the above. In addition, a part of the above steps may be executed simultaneously (in parallel) with other steps.

[0598] In the above, the three-dimensional data encoding device and the three-dimensional data decoding device related to one or more technical solutions are described based on the implementation mode, but the present invention is not limited to the implementation mode. As long as it does not deviate from the main purpose of the present invention, various deformation forms that can be thought of by those skilled in the art to the present embodiment, or forms constructed by combining the constituent elements of different implementation modes can also be included in the scope of one or more technical solutions.

[0599] Industrial Applicability

[0600] The present invention can be applied to a three-dimensional data encoding device and a three-dimensional data decoding device.

[0601] Description of symbols

[0602] 4601 Three-dimensional data encoding system

[0603] 4602 3D data decoding system

[0604] 4603 Sensor Terminal

[0605] 4604 External connection

[0606] 4611 Point Group Data Generation System

[0607] 4612 Prompt Department

[0608] 4613 Coding Department

[0609] 4614 Multiplexing Department

[0610] 4615 Input and Output

[0611] 4616 Control Department

[0612] 4617 Sensor information acquisition unit

[0613] 4618 Point Group Data Generation Department

[0614] 4621 Sensor information acquisition unit

[0615] 4622 Input and Output

[0616] 4623 Inverse Multiplexing Department

[0617] 4624 Decoding Department

[0618] 4625 Prompt Department

[0619] 4626 User Interface

[0620] 4627 Control Department

[0621] 4630 1st Coding Department

[0622] 4631 Position Information Coding Unit

[0623] 4632 Attribute Information Coding Unit

[0624] 4633 Additional Information Coding Department

[0625] 4634 Multiplexing Department

[0626] 4640 Decoding Unit 1

[0627] 4641 Inverse Multiplexing Department

[0628] 4642 Position information decoding unit

[0629] 4643 Attribute information decoding unit

[0630] 4644 Additional information decoding unit

[0631] 4650 2nd Coding Department

[0632] 4651 Additional Information Generation Department

[0633] 4652 Position Image Generation Unit

[0634] 4653 Attribute Image Generation Unit

[0635] 4654 Image Coding Department

[0636] 4655 Additional Information Coding Department

[0637] 4656 Multiplexing Department

[0638] 4660 Decoding Unit 2

[0639] 4661 Inverse Multiplexing Unit

[0640] 4662 Image Decoding Unit

[0641] 4663 Additional information decoding unit

[0642] 4664 Position Information Generation Unit

[0643] 4665 Attribute Information Generation Unit

[0644] 4670 Coding Department

[0645] 4671 Multiplexing Department

[0646] 4680 Decoding Department

[0647] 4681 Inverse Multiplexing Department

[0648] 4710 No.1 Multiplexing Department

[0649] 4711 File Conversion Department

[0650] 4720 1st Inverse Multiplexing Unit

[0651] 4721 File Inverse Transformation Unit

[0652] 4730 Second Multiplexing Unit

[0653] 4731 Document Conversion Department

[0654] 4740 Second Inverse Multiplexing Unit

[0655] 4741 File Inverse Transformation Unit

[0656] 4750 No.3 Multiplexing Department

[0657] 4751 Document Conversion Department

[0658] 4760 3rd Inverse Multiplexing Unit

[0659] 4761 File Inverse Transformation Unit

[0660] 4801 Coding Department

[0661] 4802 Multiplexing Department

[0662] 5010 1st Coding Department

[0663] 5011 Division

[0664] 5012 Position Information Coding Unit

[0665] 5013 Attribute Information Coding Unit

[0666] 5014 Additional Information Coding Department

[0667] 5015 Multiplexing Department

[0668] 5020 Decoding Unit 1

[0669] 5021 Inverse Multiplexing Unit

[0670] 5022 Position information decoding unit

[0671] 5023 Attribute information decoding unit

[0672] 5024 Additional information decoding unit

[0673] 5025 Joint

[0674] 5031 Tile Division

[0675] 5032 Position information slice segmentation unit

[0676] 5033 Attribute Information Slicing Division

[0677] 5041 Position information slice combination unit

[0678] 5042 Attribute information slice combination

[0679] 5043 Tile joint

[0680] 5051 Tile Division

[0681] 5052 Coding Department

[0682] 5053 Decoding Department

[0683] 5054 Tile joint

Claims

1. A three-dimensional data encoding method, wherein: By encoding a plurality of subspaces obtained by dividing an object space including a plurality of three-dimensional points, a plurality of encoded data are generated. A bit stream including first information indicating whether the plurality of subspaces overlap is generated.

2. The three-dimensional data encoding method according to claim 1, wherein: The bit stream includes second information indicating whether the division method of the plurality of subspaces is a division method using a bird's-eye view.

3. The three-dimensional data encoding method according to claim 1 or 2, wherein: The bitstream includes third information indicating at least one of height, width, depth, and radius of the plurality of subspaces.

4. The three-dimensional data encoding method according to claim 1 or 2, wherein: The bit stream includes fourth information, and the fourth information indicates the position of each of the plurality of subspaces.

5. The three-dimensional data encoding method according to claim 1 or 2, wherein: The bit stream includes fifth information indicating the number of the plurality of subspaces.

6. The three-dimensional data encoding method according to claim 1 or 2, wherein: The bit stream includes sixth information, and the sixth information indicates intervals between the plurality of subspaces.

7. A three-dimensional data decoding method, wherein: The plurality of subspaces are restored by decoding a plurality of coded data generated by encoding a plurality of subspaces divided into a target space including a plurality of three-dimensional points and included in the bit stream, The bit stream includes first information indicating whether the plurality of subspaces overlap.

8. The three-dimensional data decoding method according to claim 7, wherein: The bit stream includes second information indicating whether the division method of the plurality of subspaces is a division method using a bird's-eye view.

9. The three-dimensional data decoding method according to claim 7 or 8, wherein: The bitstream includes third information indicating at least one of height, width, depth, and radius of the plurality of subspaces.

10. The three-dimensional data decoding method according to claim 7 or 8, wherein: The bit stream includes fourth information, and the fourth information indicates the position of each of the plurality of subspaces.

11. The three-dimensional data decoding method according to claim 7 or 8, wherein: The bit stream includes fifth information indicating the number of the plurality of subspaces.

12. The three-dimensional data decoding method according to claim 7 or 8, wherein: The bit stream includes sixth information, and the sixth information indicates intervals between the plurality of subspaces.

13. The three-dimensional data decoding method according to claim 7 or 8, wherein: When the plurality of subspaces include two overlapping subspaces, processing is performed using a plurality of data obtained by decoding the encoded data corresponding to the two subspaces.

14. The three-dimensional data decoding method according to claim 13, wherein: The above processing includes smoothing or filtering.

15. A three-dimensional data encoding device, wherein: have: Processor; and Memory; The processor uses the memory. By encoding a plurality of subspaces obtained by dividing an object space including a plurality of three-dimensional points, a plurality of encoded data are generated. A bit stream including first information indicating whether the plurality of subspaces overlap is generated.

16. A three-dimensional data decoding device, wherein: have: Processor; and Memory; The processor uses the memory. The plurality of subspaces are restored by decoding a plurality of coded data generated by encoding a plurality of subspaces divided into a target space including a plurality of three-dimensional points and included in the bit stream, The bit stream includes first information indicating whether the plurality of subspaces overlap.

Images