Encoding method, decoding method, and information processing system

Abstract

This encoding method includes: acquiring point cloud data including header data and body data; extracting the header data from the point cloud data; and encoding the body data. Encoding includes generating encoded point cloud data including the header data, and the body data after encoding.

Description

Encoding method, decoding method, and information processing system

[0001] The present disclosure relates to an encoding method, a decoding method, and an information processing system.

[0002] Various techniques have been proposed for encoding and decoding three-dimensional point group data, also called point clouds (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2023-123508

[0004] The point cloud data may also contain header data that describes additional information about the point cloud data. Conventional encoding and decoding techniques do not target header data, and therefore there is a problem in that information in the header data contained in the point cloud data before encoding is lost.

[0005] One aspect of the present disclosure is to ensure that information in the header data is not lost.

[0006] An encoding method according to one aspect of the present disclosure includes obtaining point cloud data including header data and body data, extracting the header data from the point cloud data, and encoding the body data, wherein the encoding includes generating encoded point cloud data including the header data and the encoded body data.

[0007] A decoding method according to one aspect of the present disclosure includes obtaining encoded point cloud data, which is data obtained by encoding point cloud data including header data and body data, extracting the header data from the encoded point cloud data, decoding the encoded point cloud data to obtain the body data, and generating point cloud data including the header data and the body data.

[0008] An information processing system according to one aspect of the present disclosure includes an encoding device that encodes point cloud data including header data and body data, and a decoding device that decodes encoded point cloud data, which is the point cloud data after being encoded by the encoding device. The encoding by the encoding device includes extracting header data from the point cloud data, encoding the body data, and generating encoded point cloud data including the header data and the body data after encoding. The decoding by the decoding device includes extracting the header data from the encoded point cloud data, decoding the encoded point cloud data to obtain the body data, and generating point cloud data including the header data and the body data.

[0009] 1 is a diagram illustrating an example of a schematic configuration of an information processing system 3 according to an embodiment. FIG. 1 is a diagram illustrating an example of point cloud data d1. FIG. 2 is a diagram illustrating an example of header data d2. FIG. 2 is a diagram illustrating an example of a schematic configuration of an encoding device 1. FIG. 3 is a diagram illustrating an example of search information 102. FIG. 4 is a diagram illustrating an example of encoded point cloud data d8. FIG. 5 is a flowchart illustrating an example of a process (encoding method) executed in the encoding device 1. FIG. 6 is a flowchart illustrating an example of a process (encoding method) executed in the encoding device 1. FIG. 7 is a flowchart illustrating an example of a process (encoding method) executed in the encoding device 1. FIG. 8 is a flowchart illustrating an example of a process (encoding method) executed in the encoding device 1. FIG. 9 is a diagram illustrating an example of a schematic configuration of a decoding device 2. FIG. 10 is a flowchart illustrating an example of a process (decoding method) executed in the decoding device 2. FIG. 11 is a flowchart illustrating an example of a process (decoding method) executed in the decoding device 2. FIG. 12 is a diagram illustrating an example of a process (encoding method) executed in the encoding device 1. FIG. 13 is a diagram illustrating an example of a process (decoding method) executed in the decoding device 2. FIG. 14 is a diagram illustrating an example of a process (encoding method) executed in the encoding device 1. FIG. 15 is a diagram illustrating an example of a process (decoding method) executed in the decoding device 2. Fig. 1 is a diagram showing an example of the arrangement of header data d2. Fig. 2 is a diagram showing examples of header data d2 and search information 102 corresponding to the LAS format. Fig. 3 is a diagram showing examples of header data d2 and search information 102 corresponding to the STL format. Fig. 4 is a diagram showing an example of the hardware configuration of the device.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same elements are designated by the same reference numerals, and redundant description will be omitted.

[0011] The present disclosure will be described in the following order: 1. Embodiment 2. Modification 2.1 Compression of header data 2.2 Other examples of header data arrangement 2.3 Other examples of point cloud data format 2.4 Other examples of encoding techniques 3. Example of device hardware configuration 4. Summary

[0012] 1. Embodiment Fig. 1 is a diagram showing an example of a schematic configuration of an information processing system 3 according to an embodiment. The information processing system 3 includes an encoding device 1 and a decoding device 2. The encoding device 1 encodes data. The encoded data is also referred to as encoded data. The decoding device 2 decodes the encoded data.

[0013] Note that "information" may be interpreted as indicating the meaning of "data" and may be interpreted as having the same meaning as "data." "Information" and "data" may be interpreted as appropriate, provided there is no contradiction. "Encoding" may be interpreted as meaning "compression," and these may be interpreted as appropriate, provided there is no contradiction. Encoded data is also referred to as encoded data. "Decoding" may be interpreted as meaning "decompression," and these may be interpreted as appropriate, provided there is no contradiction.

[0014] In this embodiment, the encoding device 1 encodes point cloud data d1. The point cloud data d1 is three-dimensional point cloud data, also referred to as a point cloud. The point cloud data d1 may be described in various known data formats. Examples of data formats include the PLY (Polygon File) format, the LAS (LiDAR Data Exchange) format, and the STL (Standard Triangulated Language) format. Unless otherwise specified, in this embodiment, the point cloud data d1 is described in the PLY format.

[0015] 2 is a diagram showing an example of point cloud data d1. The point cloud data d1 includes header data d2 and body data d3.

[0016] The header data d2 is data that is added to the point cloud data d1 separately from the body data d3, and describes, for example, various information related to the body data d3. The description will be made with reference to FIG.

[0017] 3 is a diagram showing an example of the header data d2. The header data d2 corresponding to the PLY format is shown in schematic form. Examples of data included in the header data d2 include format data, comment data, variable type data, and variable name data.

[0018] The format data describes information about the format of the body data d3, such as the type of format. Figure 3 shows two formats, ascii and binary, as examples. The comment data describes arbitrary information, such as information about the software used to generate the body data d3, the date and time the data was generated, etc. Figure 3 shows the comment as a schematic representation of xxx.

[0019] The variable type data describes information on the types of various variables used in the body data d3, such as the type of the variable. The variable name data describes the names of various variables used in the body data d3. In Fig. 3, double and float are exemplified as variable types. x, y, z, confidence, and nx are exemplified as variable names.

[0020] 2, the body data d3 is data that describes the specific contents of the point cloud. The illustrated body data d3 is data that can be separated into geometry data d31 and attribute data d32 during encoding (more specifically, G-PCC, which will be described later), and is illustrated as including these.

[0021] The geometry data d31 is position data indicating the position of each point in the point cloud data d1, and specifies each point in an xyz coordinate system, for example. The attribute data d32, as its name suggests, indicates attribute data. The attribute data d32 is associated (linked) with the geometry data d31, and can thereby indicate attribute data for each position in the point cloud data d11. Note that multiple pieces of attribute data d32 may be included in the body data d3, in which case each piece of attribute data d32 has a different attribute and indicates data for that attribute.

[0022] Examples of attributes are color, reflectance, normal, class, etc. For example, attribute data d32 having a color attribute indicates the ratio of red (R), green (G), blue (B), etc. Attribute data d32 having a reflectance attribute indicates reflectance. Attribute data d32 having a normal attribute indicates the orientation of the surface corresponding to each point. A class is a class (also called a label, etc.) obtained by classification. Attribute data d32 having a class attribute indicates the probability (likelihood) of belonging to that class. An example of class classification is, but is not limited to, object classification.

[0023] Any other attribute other than those described above may be defined and used. One example of another attribute is the degree of unnecessaryness. The attribute data d32 having the attribute of the degree of unnecessaryness indicates the degree of unnecessaryness of an object located at each point or each portion. For example, this can be used to prevent an object with a low degree of unnecessaryness from being presented as an image or video. In addition, there may be attribute data d32 having a wide variety of attributes, such as those used in 3D Gaussian splatting.

[0024] 1 , for example, the above-described point cloud data d1 is encoded by the encoding device 1. The encoded point cloud data d1 is referred to as encoded point cloud data d8. As will be described in detail later, the encoded point cloud data d8 is configured so that not only the body data d3 but also the header data d2 can be restored.

[0025] The encoded point cloud data d8 is sent from the encoding device 1 to the decoding device 2. The encoded point cloud data d8 may be sent (transmitted) from the encoding device 1 to the decoding device 2 via a communication network, or may be sent (transported) from the encoding device 1 to the decoding device 2 using a recording medium.

[0026] The decoding device 2 decodes the encoded point cloud data d8, thereby obtaining point cloud data d1 in the decoding device 2. As described above with reference to FIG. 2, this point cloud data d1 includes header data d2 and body data d3.

[0027] In the information processing system 3, the information of the header data d2 included in the point cloud data d1 is not lost even after the encoding by the encoding device 1 and the decoding by the decoding device 2. Specific techniques for achieving this will be described below in the order of the encoding device 1 and the decoding device 2.

[0028] <Encoding device 1> Fig. 4 is a diagram showing an example of a schematic configuration of the encoding device 1. The encoding device 1 includes a storage unit 10, an acquisition unit 11, an extraction unit 12, an encoding unit 13, and a generation unit 14. In this example, the generation unit 14 is included in the encoding unit 13 as one of its components.

[0029] The storage unit 10 stores data used by the encoding device 1. Examples of data stored in the storage unit 10 include a program 101 and search information 102. The program 101 is a program (software) for causing a computer to function as the encoding device 1. The program 101 can also be called an application program (application software) for causing a computer to execute various processes by the acquisition unit 11, extraction unit 12, encoding unit 13, and generation unit 14. The search information 102 will be described later.

[0030] The acquisition unit 11 acquires the point cloud data d1. The acquisition may be performed by reading, inputting, receiving, or the like of the point cloud data d1, and does not exclude other acquisition modes.

[0031] The extraction unit 12 extracts the header data d2 from the point cloud data d1. As described above, the point cloud data d1 includes the header data d2 and the body data d3, and the extraction unit 12 extracts the header data d2.

[0032] The extraction unit 12 may extract the header data d2 from the point cloud data d1 by referring to the search information 102. The search information 102 describes information for searching for data to be extracted. The search information 102 is, for example, generated in advance and stored in the storage unit 10. The search information 102 will be described with reference to FIG. 5 as well.

[0033] 5 is a diagram showing an example of the search information 102. The search information 102 corresponding to the PLY format is shown in schematic form. Examples of information included in the search information 102 include search information for format data, search information for comment data, search information for variable type data, and search information for variable name data.

[0034] In this example, the search information 102 is information about character strings, and the character strings are described in list format. The search information for format data is exemplified by the character string "format." The search information for comment data is exemplified by the character string "comment." The search information for variable type data and the search information for variable name data is exemplified by the character string "property."

[0035] For example, the extraction unit 12 extracts the header data d2 from the point cloud data d1 by referring to the search information 102. For example, the extraction unit 12 searches the point cloud data d1 for a character string indicated in the search information 102, and acquires the character string that is found in the search and the character string that follows it. Acquiring this character string corresponds to extracting the header data d2.

[0036] 4 , for example, the extraction unit 12 extracts header data d2 from point cloud data d1 as described above. The data (remaining data) of point cloud data d1 that was not extracted by the extraction unit 12 corresponds to body data d3. In other words, the header data d2 and body data d3 included in point cloud data d1 are separated by the extraction unit 12.

[0037] The encoding unit 13 encodes the body data d3. An example of the encoding performed by the encoding unit 13 is G-PCC (Geometry-based point cloud compression). The following description will be given assuming that the encoding is G-PCC. Note that, before the encoding unit 13 encodes the body data d3, various preprocessing processes may be performed on the body data d3. An example of the preprocessing is data division (slice / tile division).

[0038] The encoding unit 13 includes a spatial quantization unit 131, an octree encoding unit 132, an attribute data conversion unit 133, an arithmetic encoding unit 134, a generation unit 14, and a syntax encoding unit 135. In this example, the syntax encoding unit 135 is also a component of the generation unit 14.

[0039] The spatial quantization unit 131 spatially quantizes the body data d3. The spatial quantization unit 131 divides (the target space of) the body data d3 into voxels of uniform size and converts coordinates into integers. The body data d3 after spatial quantization is referred to as spatially quantized data d4.

[0040] The octree encoding unit 132 performs octree encoding on the spatial quantized data d4. Each voxel of the spatial quantized data d4 is divided into eight voxels. The spatial quantized data d4 after octree encoding is referred to as octree data d5. Note that the number of voxel divisions by the octree encoding unit 132 may be a number other than eight, and in this sense, "octree" may simply be read as "tree."

[0041] The attribute data conversion unit 133 performs data conversion to obtain attribute data d32 associated with the geometry data d31. This conversion is performed based on the octree data d5, etc. The attribute data d32 obtained in this manner is referred to as converted attribute data d6.

[0042] The arithmetic coding unit 134 arithmetically codes the octree data d5 and the converted attribute data d6. An example of arithmetic coding is entropy coding. The data after arithmetic coding is referred to as arithmetically coded data d7. The arithmetically coded data d7 can also be said to be body data d3 after coding up to the arithmetic coding.

[0043] The generation unit 14 generates data including header data d2 and arithmetically coded data d7 (body data d3 after coding). This generated data is referred to as coded point group data d8. Specifically, the syntax coding unit 135 syntax-codes the arithmetically coded data d7, and the generation unit 14 adds the header data d2 thereto. The coded point group data d8 will be described with reference to FIG. 6 as well.

[0044] 6 is a diagram showing an example of encoded point group data d8. In this example, the encoded point group data d8 has a G-PCC syntax structure. The encoded point group data d8 includes multiple units. Examples of the multiple units include an AU (Access unit) header, a UD (User data data unit), an SPS (Sequence parameter set data unit), a GPS (Geometry parameter set data unit), a GDU (Geometry data unit), an APS (Attribute parameter set data unit), and an ADU (Attribute data unit). When no particular distinction is made between these, they are also simply referred to as units.

[0045] Data related to the arithmetically coded body data d3 is mainly included in GPS, GDU, APS, and ADU. GPS and GDU are related to geometry data d31. For example, GPS includes data related to parameters of the geometry data d31. GDU includes arithmetically coded geometry data d31. APS and ADU are related to attribute data d32. For example, APS includes data related to parameters of the attribute data d32. ADU includes arithmetically coded attribute data d32.

[0046] The generator 14 includes (places) the header data d2 in one of the multiple units. In this example, the generator 14 includes the header data d2 in the UD. Specifically, the UD includes a user data header and a user data payload. The header data d2 is included in the user data payload.

[0047] The user data payload, for example, assigns an ID to data in byte units and describes the contents of the data. The user data payload can include N pieces of data (N is an integer equal to or greater than 1). The ID of each piece of data is shown as user data ID_0 to ID_N, and the corresponding data contents are shown as user data byte ID_0 to user data byte ID_N. For example, the data in the first row of header data d2 can correspond to user data ID_0 and user data byte ID_0. The same applies to the data in the second row and subsequent rows.

[0048] Returning to Fig. 4, for example, the encoded point cloud data d8 as described above is generated by the generation unit 14. The generated encoded point cloud data d8 may be output as appropriate. The output may be data readout, transmission, etc., and other output modes are not excluded. An example of output is the transmission of the encoded point cloud data d8 from the encoding device 1 to the decoding device 2, as previously described with reference to Fig. 1.

[0049] The encoded point cloud data d8 obtained by encoding the point cloud data d1 by the encoding device 1 includes the same header data d2 as the point cloud data d1. Therefore, the information in the header data d2 is not lost even after encoding by the encoding device 1. For example, not only the information in the body data d3 included in the point cloud data d1 but also the information in the header data d2 can be sent from the encoding device 1 to the decoding device 2. The information in the header data d2 is not lost even after decoding by the decoding device 2, as will be described later.

[0050] 7 to 10 are flowcharts showing examples of processing (encoding methods) executed in the encoding device 1. Descriptions of content that overlap with those described above will be omitted where appropriate.

[0051] 7 shows the overall flow. In step S11, the acquisition unit 11 acquires point cloud data d1. In step S12, the extraction unit 12 extracts header data d2 from the point cloud data d1. In other words, the header data d2 and body data d3 are separated from the point cloud data d1.

[0052] In step S13, the spatial quantization unit 131 spatially quantizes the body data d3. Spatial quantized data d4 is obtained. In step S14, the octree encoding unit 132 octree-encodes the spatial quantized data d4. Octree data d5 is obtained. In step S15, the attribute data conversion unit 133 performs conversion based on the octree data d5, etc. Converted attribute data d6 is obtained. In step S16, the arithmetic encoding unit 134 arithmetically encodes the octree data d5 and the converted attribute data d6. Arithmetically encoded data d7 is obtained. As mentioned above, the arithmetically encoded data d7 is the body data d3 after encoding up to the arithmetic encoding.

[0053] In step S17, the generation unit 14 syntax-encodes the arithmetically coded data d7 and the header data d2, and generates coded point group data d8 including them.

[0054] FIG. 8 shows details of step S12. In step S121, the extraction unit 12 reads the point cloud data d1. In step S122, the extraction unit 12 refers to the search information 102 to search the point cloud data d1. The character string indicated in the search information 102 is searched for in the point cloud data d1. In step S123, the extraction unit 12 extracts the data found in the search as header data d2. For example, the character string found in the search and the character string following it are acquired as the header data d2. The specific procedures of steps S122 and S123 are not particularly limited, but a repetitive process using a While statement may be used as an example.

[0055] 9 shows the details of step S17. In steps S171 to S176, the generation unit 14 performs syntax encoding so as to obtain encoded point cloud data d8 including UD, SPS, GPS, GDU, APS, and ADU. One of the features is the UD encoding in step S171.

[0056] 10 shows the details of step S171. In step S1711, the generation unit 14 reads the header data d2 extracted by the extraction unit 12 in the previous step S12. In step S1712, the generation unit 14 includes the header data d2 in the UD. As described above with reference to FIG. 6, IDs and data contents, the same number as the number of data, are added to the user data payload of the UD. The specific procedure of step S1712 is not particularly limited, but a repetitive process using a While statement may be used as an example.

[0057] For example, the above-described exemplary process is executed, and encoded point cloud data d8, which is data obtained by encoding the point cloud data d1, is obtained.

[0058] 11 is a diagram illustrating an example of a schematic configuration of the decoding device 2. The decoding device 2 includes a storage unit 20, an acquisition unit 21, an extraction unit 22, a decoding unit 23, and a generation unit 24. In this example, the extraction unit 22 is included in the decoding unit 23 as one of its components.

[0059] The storage unit 20 stores data used by the decoding device 2. An example of data stored in the storage unit 20 is a program 201. The program 201 is a program (software) for causing a computer to function as the decoding device 2. The program 201 can also be called an application program (application software) for causing a computer to execute various processes by the acquisition unit 21, extraction unit 22, decoding unit 23, and generation unit 24.

[0060] The acquisition unit 21 acquires the encoded point group data d8. The acquisition may be performed by reading or inputting the encoded point group data d8, and does not exclude other acquisition modes.

[0061] The decoding unit 23 extracts header data d2 from the encoded point cloud data d8 and decodes the encoded point cloud data d8 to obtain body data d3. The decoding unit 23 includes an extraction unit 22, a spatial inverse quantization unit 231, an octree decoding unit 232, an attribute data inverse conversion unit 233, an arithmetic decoding unit 234, and a syntax decoding unit 235. Unless otherwise specified, the processing by the spatial inverse quantization unit 231 to the syntax decoding unit 235 may be considered to be the reverse of the processing by the spatial quantization unit 131 to the syntax coding unit 135 ( FIG. 4 ) of the encoding device 1 described above. In this example, the syntax decoding unit 235 is also a component of the extraction unit 22.

[0062] The extraction unit 22 includes a syntax encoding unit 135, and syntax-decodes the encoded point cloud data d8 and extracts the header data d2 from the encoded point cloud data d8. For example, as described above with reference to Fig. 6, the header data d2 is included in the user data payload, and the header data d2 is extracted from there.

[0063] Of the encoded point group data d8, data (remaining data) that has not been extracted by the extraction unit 22 becomes the arithmetically coded data d7. In other words, the arithmetically coded data d7 and the header data d2 included in the encoded point group data d8 are separated by the extraction unit 22.

[0064] The arithmetic decoding unit 234 arithmetically decodes the arithmetically coded data d7, thereby obtaining octree data d5 and converted attribute data d6.

[0065] The attribute data inverse conversion unit 233 performs inverse conversion of data to obtain the attribute data d32. This inverse conversion is performed based on the octree data d5, the converted attribute data d6, and the like.

[0066] The octree decoding unit 232 performs octree decoding based on the octree data d5 and the result of the inverse conversion by the attribute data inverse conversion unit 233. As a result, spatial quantized data d4 is obtained.

[0067] The spatial inverse quantization unit 231 spatially inverse quantizes the spatial quantized data d4, thereby obtaining body data d3.

[0068] The generator 24 generates data including header data d2 and body data d3. This data corresponds to the point cloud data d1, which is the encoded point cloud data d8 before encoding, and is therefore also referred to as point cloud data d1. It can be said that the point cloud data d1 is reconstructed by the generator 24.

[0069] The point cloud data d1 generated by the generating unit 24 may be output at any time. The output may be data readout, transmission, or the like, and does not exclude other output modes.

[0070] The point cloud data d1 obtained by decoding the encoded point cloud data d8 by the decoding device 2 includes header data d2, just like the point cloud data d1 that was the subject of encoding by the previous encoding device 1 (FIG. 4). Therefore, the information in the header data d2 is not lost even after decoding by the decoding device 2. Not only the information in the body data d3 but also the information in the header data d2 can be restored and used.

[0071] 12 to 15 are flowcharts showing examples of the processing (decoding method) executed in the decoding device 2. Description of content that overlaps with the above will be omitted where appropriate.

[0072] 12 shows the overall flow. In step S21, the acquisition unit 21 acquires encoded point cloud data d8. In step S22, the extraction unit 22 decodes the encoded point cloud data d8 and extracts the header data d2. In other words, the header data d2 and the arithmetically coded data d7 are separated from the encoded point cloud data d8.

[0073] In step S23, the arithmetic decoding unit 234, the attribute data inverse conversion unit 233, the octree decoding unit 232, and the spatial inverse quantization unit 231 decode the arithmetically coded data d7 to obtain body data d3. In step S24, the generation unit 24 generates point cloud data d1 including the header data d2 and the body data d3.

[0074] 13 shows the details of step S22. In step S221, the extraction unit 22 reads the encoded point group data d8. In steps S222 to S227, the extraction unit 22 decodes the UD, SPS, GPS, GDU, APS, and ADU, and extracts the header data d2. One of the features is the UD decoding in step S222.

[0075] 14 shows the details of step S222. In step S2221, the extraction unit 22 decodes the UD, reads the user data payload, and extracts it as header data d2. As described above with reference to FIG. 6, the header data d2 is included in the user data payload of the UD, and this data is read out. For example, a loop process using a While statement may be used.

[0076] FIG. 15 shows the details of step S23. In step S231, the arithmetic decoding unit 234 arithmetically decodes the arithmetically coded data d7. Octree data d5 and transformed attribute data d6 are obtained. In step S232, the attribute data inverse conversion unit 233 performs inverse conversion based on the octree data d5, the transformed attribute data d6, etc. In step S233, the octree decoding unit 232 performs octree decoding based on the octree data d25 and the result of the inverse conversion. Spatial quantized data d4 is obtained. In step S234, the spatial inverse quantization unit 231 spatially inverse quantizes the spatial quantized data d4. Body data d3 is obtained.

[0077] For example, the above-described exemplary process is executed, and point group data d1, which is data obtained by decoding the encoded point group data d8, is obtained.

[0078] 2. Modifications The disclosed technology is not limited to the above-described embodiment. Some modifications will be described.

[0079] 2.1 Compression of Header Data In one embodiment, the header data d2 may be compressed and included in the encoded point cloud data d8. Note that the process of compressing the header data d2 here is a process separate from the syntax encoding process. One example of compression is ZIP compression, which is effective for compressing header data d2 that contains a large amount of character string information, for example. Differences from the encoding and decoding process flows described above will be described with reference to FIGS. 16 and 17.

[0080] 16 is a diagram showing an example of processing (encoding method) executed in the encoding device 1. The processing of step S171a shown in the figure is executed instead of the processing of step S171 described above in FIGS. 9 and 10. Compared to step S171 (FIG. 11), step S171a includes step S1712a instead of step S1712.

[0081] In step S1712a, the generation unit 14 compresses the header data d2 and includes it in the UD. An example of compression is ZIP compression, but this is not limiting. Thereafter, the processes of steps S172 to S176 in FIG. 9 described above are executed. In this way, the generation unit 14 generates coded point cloud data d8 including the compressed header data d2 and the arithmetically coded data d7 (coded body data d3).

[0082] Fig. 17 is a diagram showing an example of processing (decoding method) executed in the decoding device 2. The processing of step S222a shown in the figure is executed instead of step S222 described above in Fig. 12 and Fig. 13. Compared to step S222 (Fig. 13), step S222a includes step S2221a instead of step S2221.

[0083] In step S2221a, the extraction unit 22 decodes the UD, reads the user data payload, decompresses it, and extracts it as header data d2. In other words, the header data d2 compressed in the previous step S1712a (FIG. 16) is decompressed and read out.

[0084] For example, as described above, the header data d2 can be compressed when encoding the point cloud data d1 to obtain the encoded point cloud data d8, and the header data d2 can be decompressed when decoding the encoded point cloud data d8 to obtain the point cloud data d1. The data size of the encoded point cloud data d8 can be reduced by the amount of compression of the header data d2. For example, when transmitting the encoded point cloud data d8 from the encoding device 1 to the decoding device 2 ( FIG. 1 ), the communication load at that time can be reduced and the time required for communication can be shortened.

[0085] 2.2 Other Arrangement Examples of Header Data In the above embodiment, the header data d2 is arranged (included) in the UD of the encoded point group data d8. However, the header data d2 may be arranged in a location other than the UD. Some examples will be described with reference to FIGS. 18 to 20.

[0086] 18 to 20 are diagrams showing examples of the arrangement of header data d2. In the example shown in Fig. 18, the header data d2 is arranged in an AU header. In the example shown in Fig. 19, the header data d2 is arranged in a NAL (Network Abstraction Layer) header. The NAL header is also one of the units included in the coded point group data d8.

[0087] In the example shown in FIG. 20 , header data d2 is placed in a unit (separate unit) that exists separately from the syntax structure of G-PCC. The data of the separate unit is described, for example, as a system format higher than G-PCC. A data unit (which may also be called a Meta Data Unit, etc.) that describes the format type, encoding type, number of attributes, attribute names, attribute data type, etc. of the encoded point cloud data d8 may be used as the separate unit. An example of a format type is Ply, etc. Examples of encoding types are Ascii, Binary, etc. The number of attributes is the number of attribute data d32 included in the body data d3 of the point cloud data d1. If there are two or more attribute data d32 each having different attributes, the number of attributes will also be two or more. The attribute name indicates, for example, the type of attribute. Examples of attribute data types are char, short, int, unit, float, doule, etc.

[0088] 2.3 Other Examples of Point Cloud Data Formats In the above embodiment, the PLY format has been mainly used as an example of the data format of the point cloud data d1. As mentioned above, various formats other than the PLY format, such as the LAS format and the STL format, may also be used. Examples of the header data d2 and the search information 102 corresponding to these formats will be described with reference to FIGS. 21 and 22.

[0089] 21A and 21B are diagrams showing examples of header data d2 and search information 102 corresponding to the LAS format. (A) of FIG. 21 shows the header data d2. The data included in the header data d2 are schematically shown as a File Signature, a File Source ID, and Global Encoding. The File Signature is 4-byte character string data in LASF. The File Source ID and Global Encoding are both 2-byte integer value data. (B) of FIG. 21 shows the search information 102. As information included in the search information 102, a File Signal, which is a four-byte character string called LASF, is shown, and a File Source ID and Global Encoding, each of which is a two-byte integer value, are shown schematically as 0x00.

[0090] Fig. 22 shows examples of header data d2 and search information 102 corresponding to the STL format. Header data d2 is shown in (A) of Fig. 22. The header data d2 is described as 80-byte character string data. Search information 102 is shown in (B) of Fig. 22. The search information 102 is an 80-byte character string (for example, from the beginning of the data).

[0091] For example, by using header data d2 and search information 102 corresponding to the data format of point cloud data d1 as described above, it is possible to prevent the information in the header data d2 from being lost even when encoding or decoding point cloud data d1 described in various data formats.

[0092] 2.4 Examples of Other Encoding Techniques In the above embodiment, the case where encoding is performed using G-PCC has been described as an example. However, various encoding techniques other than G-PCC may be used to encode the point cloud data d1. One example of another encoding technique is V-PCC (Video-based compression approach). The point cloud data d1 is expanded into a 2D image (texture, depth) and encoded using an existing video codec.

[0093] 23 is a diagram showing an example of a hardware configuration of an apparatus. The encoding apparatus 1 or the decoding apparatus 2 described above can be realized by, for example, a computer 1000 shown in FIG. 23.

[0094] The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected to each other via a bus 1050.

[0095] The CPU 1100 operates based on programs stored in the ROM 1300 or the HDD 1400 and controls each component. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs. Examples of the programs are the previously described program 101 (FIG. 4) or program 201 (FIG. 11).

[0096] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0097] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records programs for the encoding method and decoding method (which may also be called an information processing method) according to the present disclosure, which are an example of program data 1450.

[0098] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0099] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU receives data from input devices such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined computer-readable recording medium. Examples of the medium include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), magneto-optical recording media such as an MO (Magneto-Optical Disc), tape media, magnetic recording media, or semiconductor memory.

[0100] When computer 1000 functions as the various devices described above, CPU 1100 of computer 1000 realizes those functions by executing programs loaded onto RAM 1200. The programs may be stored in HDD 1400. CPU 1100 reads and executes program data 1450 from HDD 1400, but as another example, CPU 1100 may obtain the program from another device via external network 1550.

[0101] Each of the above components may be configured using general-purpose materials or may be configured using hardware specialized for the function of each component. Such configurations may be changed as appropriate depending on the technical level at the time of implementation.

[0102] 4. Summary The techniques described above can be specified, for example, as follows. One of the disclosed techniques is an encoding method. As described with reference to FIGS. 1 to 10 and 18 to 22, the encoding method includes acquiring point cloud data d1 including header data d2 and body data d3 (step S11), extracting the header data d2 from the point cloud data d1 (step S12), and encoding the body data d3 (steps S13 to S17). The encoding includes generating encoded point cloud data d8 including the header data d2 and the encoded body data d3 (arithmetically coded data d7) (step S17).

[0103] According to the encoding method described above, encoded point cloud data d8 including the header data d2 can be generated from the point cloud data d1, thereby preventing information from being lost in the header data d2.

[0104] 3 and the like, the header data d2 may include at least one of format data, comment data, variable type data, and variable name data. For example, it is possible to prevent information from being lost in the header data d2 including such various data.

[0105] 4, 5, 7, 8, etc., extracting the header data d2 (step S12) may include extracting the header data d2 from the point cloud data d1 by referring to the search information 102 that describes information for searching for data to be extracted (steps S122 and S123). For example, the header data d2 can be extracted in this manner.

[0106] As described with reference to Figures 6, 7, 10, 18 to 20, etc., the encoded point group data d8 includes a plurality of units, and generating the encoded point group data d8 (step S17) may include including the header data d2 in any one of the plurality of units (e.g., step S1712). The plurality of units may include at least one of a NAL header, an AU header, and a UD. For example, the header data d2 can be included in the encoded point group data d8 in this manner.

[0107] 7 and 16, generating the encoded point cloud data d8 (step S17) may include compressing the header data d2 and generating encoded point cloud data d8 including the compressed header data d2 and encoded body data d3 (step S1712a), thereby reducing the data size of the encoded point cloud data d8.

[0108] As described with reference to Fig. 4 etc., the encoding may be G-PCC. Other encoding techniques may be used, one example being V-PCC. When encoding the point cloud data d1 using various encoding techniques, it is possible to prevent the header data d2 from being lost.

[0109] 1 to 22, etc., is also one of the disclosed techniques. The decoding method includes obtaining encoded point cloud data d8, which is data obtained by encoding point cloud data d1 including header data d2 and body data d3 (step S21), extracting the header data d2 from the encoded point cloud data d8 (step S22), decoding the encoded point cloud data d8 to obtain body data d3 (step S23), and generating point cloud data d1 including the header data d2 and the body data d3 (step S24).

[0110] According to the above decoding method, it is possible to generate point cloud data d1 including header data d2 from encoded point cloud data d8, thereby preventing information in the header data d2 from being lost.

[0111] As described with reference to Figures 7, 12, 13, 16, and 17, the header data d2 included in the encoded point cloud data d8 is compressed header data d2 (step S1712a), and extracting the header data d2 (step S22) may include decompressing the compressed header data d2 (step s2221a), thereby reducing the data size of the encoded point cloud data d8.

[0112] The information processing system 3 described with reference to Figures 1 to 22 is also one of the disclosed technologies. The information processing system 3 includes an encoding device 1 that encodes point cloud data d1 including header data d2 and body data d3, and a decoding device 2 that decodes encoded point cloud data d8, which is the point cloud data d1 after encoding by the encoding device 1. The encoding by the encoding device 1 includes extracting the header data d2 from the point cloud data d1 (step S12), encoding the body data d3, and generating encoded point cloud data including the header data d2 and the encoded body data d3 (steps S13 to S17). The decoding by the decoding device 2 includes extracting the header data d2 from the encoded point cloud data d8 (step S22), decoding the encoded point cloud data d8 to obtain the body data d3, and generating point cloud data d1 including the header data d2 and the body data d3 (steps S23 and S24). As explained above, such an information processing system 3 can also prevent information in the header data d2 from being lost.

[0113] The effects described in this disclosure are merely examples and are not limited to the disclosed contents. Other effects may also be obtained.

[0114] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

[0115] Note that the present technology can also be configured as follows. (1) An encoding method including: acquiring point cloud data including header data and body data; extracting the header data from the point cloud data; and encoding the body data, wherein the encoding includes generating encoded point cloud data including the header data and the encoded body data. (2) The encoding method according to (1), wherein the header data includes at least one of format data, comment data, variable type data, and variable name data. (3) The encoding method according to (1) or (2), wherein extracting the header data includes extracting the header data from the point cloud data by referencing search information describing information for searching for data to be extracted. (4) The encoding method according to any of (1) to (3), wherein the encoded point cloud data includes a plurality of units, and generating the encoded point cloud data includes including the header data in any one of the plurality of units. (5) The encoding method according to (4), wherein the plurality of units include at least one of a Network Abstraction Layer (NAL) header, an Access Unit (AU) header, and a User Data Unit (UD). (6) The encoding method according to any one of (1) to (5), wherein generating the encoded point cloud data includes compressing the header data and generating encoded point cloud data including the compressed header data and the encoded body data. (7) The encoding method according to any one of (1) to (6), wherein the encoding includes Geometry-based point cloud compression (G-PCC). (8) The encoding method according to any one of (1) to (6), wherein the encoding includes Video-based compression approach (V-PCC).(9) A decoding method comprising: acquiring encoded point cloud data, which is data obtained by encoding point cloud data including header data and body data; extracting the header data from the encoded point cloud data; decoding the encoded point cloud data to obtain the body data; and generating point cloud data including the header data and the body data. (10) The decoding method according to (9), wherein the header data includes at least one of format data, comment data, variable type data, and variable name data. (11) The decoding method according to (9) or (10), wherein the encoded point cloud data includes a plurality of units, and the header data is included in any one of the plurality of units. (12) The decoding method according to (11), wherein the plurality of units include at least one of a Network Abstraction Layer (NAL) header, an Access Unit (AU) header, and a User Data Unit (UD). (13) The decoding method according to any one of (9) to (12), wherein the header data included in the encoded point cloud data is compressed header data, and extracting the header data includes decompressing the compressed header data. (14) The decoding method according to any one of (9) to (13), wherein the encoding includes G-PCC (Geometry-based point cloud compression). (15) The decoding method according to any one of (9) to (13), wherein the encoding includes V-PCC (Video-based compression approach).(16) An information processing system comprising: an encoding device that encodes point cloud data including header data and body data; and a decoding device that decodes encoded point cloud data, which is the point cloud data after being encoded by the encoding device; wherein the encoding by the encoding device includes extracting the header data from the point cloud data, encoding the body data, and generating encoded point cloud data including the header data and the body data after encoding; and the decoding by the decoding device includes extracting the header data from the encoded point cloud data, and decoding the encoded point cloud data to obtain the body data, and generating point cloud data including the header data and the body data.

[0116] REFERENCE SIGNS LIST 1 Encoding device 10 Storage unit 101 Program 102 Search information 11 Acquisition unit 12 Extraction unit 13 Encoding unit 131 Spatial quantization unit 132 Octree encoding unit 133 Attribute data conversion unit 134 Arithmetic encoding unit 135 Syntax encoding unit 14 Generation unit 2 Decoding device 20 Storage unit 201 Program 21 Acquisition unit 22 Extraction unit 23 Decoding unit 231 Spatial inverse quantization unit 232 Octree decoding unit 233 Attribute data inverse conversion unit 234 Arithmetic decoding unit 235 Syntax decoding unit 24 Generation unit 3 Information processing system d1 Point cloud data d2 Header data d3 Body data d31 Geometry data d32 Attribute data d4 Spatial quantization data d5 Octree data d6 Converted attribute data d7 Arithmetically coded data d8 Coded point cloud data

Claims

1. An encoding method comprising: obtaining point cloud data including header data and body data; extracting the header data from the point cloud data; and encoding the body data, wherein the encoding step comprises generating encoded point cloud data including the header data and the body data after encoding.

2. The encoding method according to claim 1, wherein the header data includes at least one of format data, comment data, variable type data, and variable name data.

3. The encoding method according to claim 1, wherein extracting the header data includes extracting the header data from the point cloud data by referring to search information that describes information for searching for data to be extracted.

4. The encoding method according to claim 1, wherein the encoded point group data includes a plurality of units, and generating the encoded point group data includes including the header data in any one of the plurality of units.

5. The encoding method of claim 4, wherein the plurality of units include at least one of a Network Abstraction Layer (NAL) header, an Access Unit (AU) header, and a User Data Unit (UD).

6. The encoding method according to claim 1, wherein generating the encoded point cloud data includes compressing the header data and generating encoded point cloud data including the compressed header data and the encoded body data.

7. The encoding method of claim 1, wherein the encoding includes G-PCC (Geometry-based point cloud compression).

8. The encoding method of claim 1, wherein the encoding includes a video-based compression approach (V-PCC).

9. A decoding method comprising: obtaining encoded point cloud data, which is data obtained by encoding point cloud data including header data and body data; extracting the header data from the encoded point cloud data; decoding the encoded point cloud data to obtain the body data; and generating point cloud data including the header data and the body data.

10. The decoding method of claim 9, wherein the header data includes at least one of: format data; comment data; variable type data; and variable name data.

11. The decoding method according to claim 9, wherein the encoded point group data includes a plurality of units, and the header data is included in one of the plurality of units.

12. The decoding method of claim 11, wherein the plurality of units include at least one of a Network Abstraction Layer (NAL) header, an Access Unit (AU) header, and a User Data Unit (UD).

13. The decoding method according to claim 9, wherein the header data included in the encoded point cloud data is compressed header data, and extracting the header data includes decompressing the compressed header data.

14. The decoding method of claim 9, wherein the encoding includes G-PCC (Geometry-based point cloud compression).

15. The decoding method of claim 9, wherein the encoding includes a video-based compression approach (V-PCC).

16. An information processing system comprising: an encoding device that encodes point cloud data including header data and body data; and a decoding device that decodes encoded point cloud data, which is the point cloud data after being encoded by the encoding device; wherein the encoding by the encoding device includes extracting the header data from the point cloud data, encoding the body data, and generating encoded point cloud data including the header data and the body data after encoding; and the decoding by the decoding device includes extracting the header data from the encoded point cloud data, decoding the encoded point cloud data to obtain the body data, and generating point cloud data including the header data and the body data.

Images