Information processing device and method

Abstract

The present disclosure relates to an information processing device and method that enable an increase in the load of decoding processing of encoded data of a point cloud to be suppressed. Position information of a point cloud representing an object of a three-dimensional shape as a collection of points is encoded in such an order that the decoding result of the encoded data of the position information is output in the order in which the attribute information about the point cloud is processed, and the encoded data is generated. In addition, the encoded data of the position information of a point cloud representing an object of a three-dimensional shape as a collection of points is decoded, and the decoding result is output in the order in which the attribute information about the point cloud is processed. The present disclosure is applicable to, for example, an information processing device, an image processing device, an encoding device, a decoding device, an electronic device, an information processing method or a program.

Description

Technical Field

[0001] The present disclosure relates to an information processing device and method, and in particular, to an information processing device and method that enable suppression of an increase in the load of decoding processing of encoded data of a point cloud. Background Art

[0002] Conventionally, a method for encoding 3D data representing a three-dimensional structure such as a point cloud has been devised (for example, refer to NPL 1). The data of the point cloud is composed of geometric data (also referred to as position information) and attribute data (also referred to as attribute information) of each point. Therefore, encoding of the point cloud is performed for each of its geometric data and its attribute data.

[0003] As a method for encoding attribute data, various methods have been proposed. For example, a technique called lifting has been proposed (for example, refer to NPL 2). In addition, a method that enables attribute data to be scalably decoded has also been proposed (for example, refer to NPL 3). In addition, although there is no lifting, a method that enables reference to decoded attribute data in LoD has been designed (for example, refer to NPL 4).

[0004] In any case, assuming that the geometric data including degradation due to compression is known, the encoding of the attribute data is performed using the positional relationship between the points. More specifically, the encoding of the attribute data is performed using the correlation between points that are close to each other in space. In order to use such correlation between points, the points in the three-dimensional space are mapped to one dimension using a Morton code, and then sorted according to the value of the Morton code. In other words, each point is arranged in Morton order. Therefore, it is ensured that the mapped points that are adjacent to each other in one dimension are close to each other in space.

[0005] Reference List

[0006] Non-patent literature

[0007] [NPL 1] R. Mekuria, IEEE Student Member, K. Blom, P. Cesar, IEEE Member, “Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video”, tcsvt_paper_submitted_february.pdf

[0008] [NPL2] Khaled Mammou, Alexis Tourapis, Jungsun Kim, Fabrice Robinet, Valery Valentin, Yeping Su, “Lifting Scheme for Lossy Attribute Encoding in TMC1”, ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42640, April 2018, San Diego, USA

[0009] [NPL3] Ohji Nakagami, Satoru Kuma, “[G-PCC] Spatial scalability support for G-PCC”, ISO / IEC JTC1 / SC29 / WG11MPEG2019 / m47352, March 2019, Geneva, Switzerland

[0010] [NPL4] Toshiyasu Sugio, “[G-PCC] Reference structure modification on attribute predicting transform in TMC13”, ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m46107, January 2019, Marrakech, Morocco Summary of the invention

[0011] Technical issues

[0012] However, the sorting process generates a risk of increasing the load of the decoding process of the encoded data of the point cloud.

[0013] The present disclosure has been devised in consideration of the above-described circumstances, and an object of the present disclosure is to enable suppression of an increase in the load of decoding processing of encoded data of a point cloud.

[0014] Solution to the problem

[0015] According to aspects of the present technology, an information processing device is an information processing device comprising an encoding unit configured to: encode position information of a point cloud in sequence, wherein the point cloud expresses an object having a three-dimensional shape as a set of points, wherein the sequence causes a decoding result of the encoded data of the position information to be output in a processing sequence of attribute information of the point cloud, and generate the encoded data.

[0016] An information processing method according to aspects of the present technology is an information processing method comprising: encoding position information of a point cloud in sequence, wherein the point cloud expresses an object having a three-dimensional shape as a set of points, wherein the sequence causes a decoding result of the encoded data of the position information to be output in a processing sequence of attribute information of the point cloud, and generating the encoded data.

[0017] According to another aspect of the present technology, an information processing device is an information processing device, which includes a decoding unit, which is configured to decode encoded data of position information of a point cloud that expresses an object with a three-dimensional shape as a collection of points, and output the decoding result in a processing order of attribute information of the point cloud.

[0018] According to another aspect of the present technology, an information processing method is an information processing method, which includes the following steps: decoding encoded data of position information of a point cloud that expresses an object with a three-dimensional shape as a set of points, and outputting the decoding result in the processing order of attribute information of the point cloud.

[0019] In an information processing device and an information processing method according to aspects of the present technology, position information of a point cloud that expresses an object with a three-dimensional shape as a collection of points is encoded and encoded data of the position information is generated in the following order, which makes the decoding result of the encoded data output in the processing order of the attribute information of the point cloud.

[0020] In an information processing device and an information processing method according to another aspect of the present technology, encoded data of position information of a point cloud expressing an object having a three-dimensional shape as a collection of points is decoded, and the decoded result is output in the processing order of attribute information of the point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] [ Figure 1 ] is a diagram for illustrating an example of the order in which geometric data is processed.

[0022] [ Figure 2 ] is a diagram used to illustrate the encoding method and decoding method of geometric data.

[0023] [ Figure 3 ] is a diagram for explaining a method when DCM is applied.

[0024] [ Figure 4 ] is a diagram for explaining a method when DCM is applied.

[0025] [ Figure 5 ] is a diagram for illustrating a method for applying DCM.

[0026] [ Figure 6 ] is a block diagram showing an example of the main configuration of an encoding device.

[0027] [ Figure 7 ] is a block diagram showing a main configuration example of a position information encoding unit.

[0028] [ Figure 8 ] is a block diagram showing a main configuration example of a point cloud generation unit.

[0029] [ Fig. 9 ] is a flowchart for illustrating an example of the process of encoding processing.

[0030] [ Fig.10 ] is a flowchart for illustrating an example of the process of position information encoding processing.

[0031] [ Fig.11 ] is a flowchart for illustrating an example of the process of point cloud generation processing.

[0032] [ Fig.12 ] is a block diagram showing an example of the main configuration of a decoding device.

[0033] [ Fig.13 ] is a block diagram showing a main configuration example of a position information decoding unit.

[0034] [ Fig.14 ] is a flowchart for illustrating an example of the process of decoding processing.

[0035] [ Fig.15 ] is a flowchart for illustrating an example of the process of position information decoding processing.

[0036] [ Fig.16 ] is a diagram for illustrating an example of a state in which a local area is being decoded.

[0037] [ Fig.17 ] is a block diagram showing a main configuration example of a computer. DETAILED DESCRIPTION

[0038] Hereinafter, modes for implementing the present disclosure (hereinafter referred to as embodiments) will be described. The description will be given in the following order.

[0039] 1. Guarantee the output order of decoding results

[0040] 2. First Embodiment (Encoding Device)

[0041] 3. Second Embodiment (Decoding Device)

[0042] 4. Appendix

[0043] <1. Guarantee of the output order of decoding results>

[0044] <Documents supporting technical content, technical terms, etc.>

[0045] The scope disclosed in the present technology includes not only the contents described in the embodiments but also the contents described in the non-patent documents listed below which are known at the time of application.

[0046] NPL 1: (as above)

[0047] NPL 2: (as above)

[0048] NPL 3: (as above)

[0049] NPL 4: (as above)

[0050] NPL 5: Sebastien Lasserre, David Flynn, “[PCC] Inference of a mode using point location direct coding in TMC3”, ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42239, January 2018, Gwangju, South Korea

[0051] In other words, the contents described in the NPL listed above will also form the basis for determining the support requirements.

[0052] <Point Cloud>

[0053] Conventionally, there is 3D data such as point clouds that express a three-dimensional structure using position information, attribute information, etc. of a point group and a mesh that is composed of vertices, edges, and faces and defines a three-dimensional shape using polygonal expressions.

[0054] For example, in the case of a point cloud, a three-dimensional structure (an object having a three-dimensional shape) is expressed as a collection of a large number of points (a point group). In other words, the data of the point cloud (also referred to as point cloud data) is composed of geometric data (also referred to as position information) and attribute data (also referred to as attribute information) of each point in the point group. Attribute data can include any kind of information. For example, color information, reflectivity information, normal information, etc. can be included in the attribute data. Therefore, since the data structure is relatively simple and a sufficiently large number of points are used at the same time, any three-dimensional structure can be expressed with sufficient accuracy.

[0055] <Quantification using voxel position information>

[0056] Since the data amount of such point cloud data is relatively large, an encoding method using voxels has been devised in order to compress the data amount due to encoding, etc. A voxel refers to a three-dimensional area for quantizing geometric data (position information).

[0057] Specifically, the three-dimensional area surrounding the point cloud is divided into small three-dimensional areas called voxels, and for each voxel, it is shown whether the voxel surrounds the point. Therefore, the position of each point is quantified in units of voxels. Therefore, by converting the point cloud data into such voxel data (also called voxel data), the increase in the amount of information can be suppressed (usually, the amount of information can be reduced).

[0058] <Octtree>

[0059] In addition, it is designed to construct an octree using such voxel data relative to geometric data. An octree is a tree structure representation of voxel data. The value of each bit of the lowest-order node of the octree indicates the presence or absence of a point for each voxel. For example, a value of "1" indicates a voxel that surrounds a point, while a value of "0" indicates a voxel that does not surround a point. In an octree, one node corresponds to eight voxels. In other words, each node of the octree is composed of 8 bits of data, and the eight bits indicate the presence or absence of a point for eight voxels.

[0060] In addition, the high-order node of the octree indicates the presence or absence of a point of an area that brings together eight voxels corresponding to the low-order nodes belonging to the high-order node. In other words, the high-order node is generated by collecting information about the voxels of the low-order nodes. When the value of the node is "0", in other words, when all the corresponding eight voxels do not surround the point, the node is deleted.

[0061] Therefore, an octree consisting of nodes whose values ​​are not "0" is constructed. In other words, the octree is able to indicate the presence or absence of a voxel point at each resolution. Due to the transformation into an octree and encoding, by decoding the position information from the highest resolution (top layer) to the desired level (resolution), the point cloud data at the resolution can be restored. Specifically, decoding can be easily performed at any resolution without decoding information of unnecessary levels (resolutions). In other words, scalability of voxels (resolutions) can be achieved.

[0062] In addition, by omitting nodes having a value of "0" as described above, the resolution of voxels in an area where a point does not exist can be reduced, and an increase in the amount of information can be further suppressed (generally, the amount of information can be further reduced).

[0063] <Promotion>

[0064] By comparison, when encoding attribute data (attribute information), it is assumed that geometric data (position information) including degradation due to encoding is known, and encoding is performed using the positional relationship between points. As a method for encoding such attribute data, a method using RAHT (Regional Adaptive Hierarchical Transform) or a method using a transform called lifting such as described in NPL2 has been designed. Applying these techniques also enables attribute data to be hierarchical in a manner similar to the octree of geometric data.

[0065] For example, in the case of boosting described in NPL 2, the attribute data of each point is encoded as a difference from a predicted value derived using the attribute data of another point. In doing so, each point is hierarchical and the difference is derived according to its hierarchical structure.

[0066] In other words, for each point's attribute data, each point is classified into a prediction point and a reference point, the attribute data of the reference point is used to derive a predicted value of the attribute data of the prediction point, and the difference between the attribute data of the prediction point and the predicted value is derived. By recursively repeating such a process relative to the reference point, the attribute data of each point is stratified.

[0067] However, this hierarchical structure is generated independently of the hierarchical structure of the geometric data (e.g., octree), and does not substantially correspond to the hierarchical structure of the geometric data. In order to restore the point cloud data, the geometric data and the attribute data must be associated with each other, and in order to do so, the geometric data and the attribute data must be decoded to the highest resolution (in other words, the lowest layer). In other words, the method of applying the boost described in NPL 2 is not suitable for scalable decoding of resolution.

[0068] <Layering for Scalable Decoding>

[0069] In contrast, the hierarchical scalable decoding adapted to the resolution described in NPL 3. In the case of the method described in NPL 3, the hierarchical attribute data is performed to match the hierarchical structure of the octree of the geometric data. Specifically, the selection of the reference point and the prediction point is performed so that when there is a point in the area corresponding to the voxel of the geometric data (when there is attribute data corresponding to the point), there is also a point in the voxel one level higher than the voxel (there is attribute data corresponding to the point). In other words, the attribute information is hierarchical according to the hierarchical structure of the octree of the geometric data.

[0070] Associating the hierarchical structure of attribute data with the hierarchical structure of geometry data enables easy restoration of point cloud data of a desired resolution without decoding down to the lowest layer. As described above, the method of applying the technology described in NPL 3 adapts to scalable decoding of resolution.

[0071] <Sort Processing>

[0072] In any case, the encoding and decoding of attribute data is performed using the correlation between points that are close to each other in space. In order to use the correlation between such points, the points in the three-dimensional space are mapped to one dimension using a Morton code, and then sorted according to the value of the Morton code. In other words, a sorting process is performed on the geometric data, and each point is arranged in a Morton order. Therefore, it is ensured that the mapped points that are adjacent to each other in one dimension are close to each other in space.

[0073] For example, in Figure 1 In the octree of geometric data shown in , nodes b to d belong to node a, points F to H belong to node b, point J belongs to node c, and points K and L belong to node d. Nodes a, b, and d are octree-encoded, and node c is encoded by applying DCM (Direct Coding Mode).

[0074] By decoding the nodes a to d surrounded by the dotted circle 11 in the encoded data of the octree constructed as described above, the points F to L surrounded by the solid square 12 are output.

[0075] In doing so, the processing starts from node a, and the processing proceeds to node b, node c, and node d. However, the processing order of the processing of node b, node c, and node d is not limited, and the processing can also be performed in parallel. In addition, since node c has been encoded by applying DCM, generally, point J is output earlier than other points.

[0076] Furthermore, although points F to H are output as the decoding results of processing node b, there is no definition of the output order of points F to H. In a similar manner, although points K and L are output as the decoding results of processing node d, there is no definition of the output order of points K and L.

[0077] Therefore, the output order of each point is as follows. It should be noted that the output order of the points shown in brackets is in no particular order.

[0078] J, ((F, G, H), (K, L))

[0079] In other words, the output order of point F to point H has no specific order, the output order of point K and point L has no specific order, and the output order of point F to point H and point K and point L has no specific order.

[0080] As described above, since the output order of the geometric data of each point has a certain degree of freedom, the output order of the geometric data may not match the processing order of the attribute data. In other words, it cannot be guaranteed that the geometric data of each point is output in the processing order of the attribute data. Therefore, in order to perform the processing of the attribute data in the above processing order, it is necessary to use the sorting processing of the points of the geometric data as described above.

[0081] However, the sorting process has a high computational cost, or in other words, the sorting process causes a large processing load. That is, the sorting process requires a large amount of processing and a large amount of processing time. Therefore, performing the sorting process generates a risk of increasing the load of the decoding process of the encoded data of the point cloud. For example, the increase in computational cost due to the sorting process may make it difficult to perform the decoding process of the encoded data of the point cloud in real time. Therefore, in order to reliably implement the real-time processing of the decoding of the encoded data of the point cloud, there is a risk of increased implementation cost.

[0082] <Guarantee of the output order of decoding results>

[0083] Taking this into account, as in Figure 2 In "Method 1" described in the top row of the table shown in , the output order of the decoded results of the geometric data is set to a predetermined order. For example, the geometric data is encoded to ensure that the output order of the decoded results of the geometric data is a predetermined order. Therefore, since the point sorting process is no longer required, the increase in the load of the decoding process of the encoded data of the point cloud can be suppressed. Therefore, the increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud can be suppressed.

[0084] In addition, for example, the encoded data of the geometric data is decoded so that the decoding result is output in a predetermined order. Therefore, since the sorting process of the points is no longer required, the increase in the load of the decoding process of the encoded data of the point cloud can be suppressed. Therefore, the increase in the implementation cost of the real-time processing for reliably realizing the decoding of the encoded data of the point cloud can be suppressed.

[0085] For example, in Figure 1 In the case of the octree of the geometric data shown in , it is guaranteed that at node b, the decoding results of points F to H are output in the order of point F → point G → point H. In addition, it is guaranteed that at node d, the decoding results of points K and L are output in the order of point K → point L. In addition, it is guaranteed that the decoding results of the points (points F to H) belonging to nodes b to d, respectively, are output in the order of node b → node d.

[0086] Therefore, the output order for each point is as follows.

[0087] J, F, G, H, K, L

[0088] In other words, the output order is fixed and becomes known. Therefore, since the sorting process of the points is no longer necessary, it is possible to suppress an increase in the load of the decoding process of the encoded data of the point cloud.

[0089] The predetermined order may be any order as long as the order becomes known when the encoded data of the geometric data is decoded. Figure 2In the "method 1-1" described in the second row from the top row in the table shown in , the predetermined order may be a Morton order. Outputting in the Morton order ensures that adjacent points in the output order are spatially adjacent to each other.

[0090] In addition, for example, as in Figure 2 In the "method 1-2" described in the third row from the top row in the table shown in , the predetermined order may be the processing order of the attribute data. In other words, the output order of the decoding result of the geometry data may be the same as the processing order of the attribute data.

[0091] For example, the geometric data (position information) of a point cloud that expresses an object having a three-dimensional shape as a collection of points can be encoded in the following order and its encoded data can be generated, which order enables the decoding result of the encoded data of the geometric data to be output in the processing order of the attribute data (attribute information) of the point cloud.

[0092] In addition, for example, in an information processing device, an encoding unit can be set, which is configured to encode geometric data (position information) of a point cloud that expresses an object having a three-dimensional shape as a collection of points and generate its encoded data in the following order, the order causing the decoding result of the encoded data of the geometric data to be output in the processing order of attribute data (attribute information) of the point cloud.

[0093] Performing encoding in this manner ensures that during decoding, the geometric data of the decoding result is output in the same order as the processing order of the attribute data. Therefore, since point sorting processing is no longer required, an increase in the load of the decoding processing of the encoded data of the point cloud can be suppressed. Therefore, an increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud can be suppressed.

[0094] It should be noted that the method of encoding the geometric data is arbitrary. For example, when the geometric data is encoded by applying CABAC (Context-based Adaptive Binary Arithmetic Coding), the encoded data is decoded in the same order as the encoding order because the context is used. Therefore, in this case, encoding the geometric data in the processing order of the attribute data ensures that during decoding, the geometric data of the decoding result is output in the same order as the processing order of the attribute data.

[0095] In addition, for example, encoded data of position information of a point cloud expressing an object having a three-dimensional shape as a collection of points may be decoded, and the decoded result thereof may be output in the processing order of attribute information of the point cloud.

[0096] For example, in an information processing device, a decoding unit may be provided that is configured to decode encoded data of position information of a point cloud that expresses an object having a three-dimensional shape as a collection of points, and output the decoding result in the order of processing attribute information of the point cloud.

[0097] Since decoding is performed in this manner so that the geometric data of the decoding result is output in the same order as the processing order of the attribute data, it is no longer necessary to perform point sorting processing. Therefore, an increase in the load of decoding processing of the encoded data of the point cloud can be suppressed. Therefore, an increase in the implementation cost of real-time processing for reliably implementing decoding of the encoded data of the point cloud can be suppressed.

[0098] It should be noted that the method of decoding the geometry data is arbitrary as long as the decoding method corresponds to the encoding method. For example, when the geometry data is encoded by applying CABAC, since the encoded data thereof is decoded using the context, the encoded data of the geometry data can be decoded according to the arrangement order thereof (in other words, the encoding order of the geometry data). Performing the decoding in this way ensures that the geometry data of the decoding result is output in the same order as the processing order of the attribute data.

[0099] It should be noted that the output order of the decoding results (in other words, the processing order of the attribute data) can be in any order. For example, the decoding results can be configured to be output in Morton order (in other words, the attribute data can be configured to be processed in Morton order). Outputting in Morton order ensures that adjacent points in the output order are spatially adjacent to each other.

[0100] In addition, the geometric data can be arranged in a tree structure. Figure 2 In the "method 1-3" described in the fourth row from the top row in the table shown in , it can be ensured that at each node of the tree structure, the output order of the decoding results is a predetermined order (for example, the processing order of the attribute data or the Morton order). For example, the geometric data arranged in the tree structure can be encoded in such an order that the decoding result at each node of the tree structure is output in the processing order of the attribute data of the point cloud. Alternatively, for example, the encoded data of the geometric data arranged in the tree structure can be decoded, and the decoding result at each node of the tree structure can be output in the processing order of the attribute data of the point cloud.

[0101] Therefore, since the need to perform point sorting processing can be eliminated even when encoding or decoding tree-structured geometric data, an increase in the load of decoding processing of encoded data of a point cloud can be suppressed.

[0102] It should be noted that the tree structure may be any tree structure. For example, the tree structure may be an octree. Alternatively, for example, the tree structure may be a KD tree.

[0103] <dcm>

[0104] In addition, as a method of encoding geometric data arranged in a tree structure, in addition to a method using a tree structure (for example, octree encoding using an octree), a method of applying DCM (Direct Coding Mode) can also be used.

[0105] For example, when a node as a processing object satisfies predetermined conditions and is determined to be sparse when the voxel data is arranged in an octree, DCM is applied to obtain the relative distance (in each of the xyz directions) from the processing object node to each leaf (point) that directly or indirectly belongs to the processing object node and encode the relative distance.

[0106] It should be noted that "a node directly belonging to..." refers to a node suspended on another node in a tree structure. For example, a node directly belonging to a processing object node refers to a node that belongs to the processing object node and is one level lower than the processing object node (a so-called child node). In addition, "a node indirectly belonging to..." refers to a node suspended on another node via another node in a tree structure. For example, a node indirectly belonging to a processing object node refers to a node that belongs to the processing object node via another node and is two or more levels lower than the processing object node (for example, a so-called grandchild node).

[0107] Therefore, the application of DCM makes it possible to omit the encoding and decoding of the intermediate nodes between the processing object node and each leaf directly or indirectly belonging to the processing object node. In other words, when encoding and decoding the processing object node, each leaf directly or indirectly belonging to the processing object node can be encoded and decoded. Therefore, the increase in the load of encoding and decoding can be suppressed.

[0108] <Guarantee of output order corresponding to DCM>

[0109] Even in such a case of applying DCM, the output order of the decoding results of the geometric data can be guaranteed to be a predetermined order. However, since the processing order of the leaves (points) to which DCM is applied depends on the processing order of the nodes as processing objects, it is difficult to control the processing order and the processing order may be different from the case of octree encoding. In other words, it is difficult to control the output order of the decoding results of the points to which DCM is applied.

[0110] Taking this into account, as in Figure 2 In “Method 1-4” described in the fifth row from the top row in the table shown in FIG. 1 , control information (transmitted from the encoding side to the decoding side) regarding the output order of the decoding results of the points to which the DCM is applied may be signaled.

[0111] For example, on the encoding side, geometric data of sparse points can be encoded by applying DCM, encoded data of the geometric data can be generated, and control information related to the output order of the decoding results of the encoded data (also referred to as DCM order information) can be generated.

[0112] In addition, for example, on the decoding side, the decoding result of the encoded data of the geometric data encoded by applying the DCM can be output in the output order indicated by the DCM order information.

[0113] Therefore, since the points to which DCM is applied are guaranteed to be output in an appropriate output order, the need to perform sorting processing of the points can be similarly eliminated even when DCM is applied and an increase in the load of the decoding processing of the encoded data of the point cloud can be suppressed.

[0114] It should be noted that the DCM sequence information may be any kind of information as long as it can indicate the output order of the decoding results of the points to which the DCM is applied. For example, the DCM sequence information may include information indicating the output order of the decoding results of the points to which the DCM is applied in order from the top. In addition, the DCM sequence information may include information indicating the output order of the decoding results of the points to which the DCM is applied by the difference from the output order of the decoding results of the points to which the DCM is applied and output immediately before. In addition, the DCM sequence information may include information indicating the output order of the decoding results of the points to which the DCM is applied by the difference from the predetermined reference order.

[0115] For example, in Figure 3 In the octree shown in FIG. , nodes a to d and points F to L are connected in the same manner as Figure 1 Node a belongs to node e. Points M and L belonging to node e are encoded and decoded by applying DCM.

[0116] Even in such a case, it is ensured that the points F to N surrounded by the solid line square 21 are output in the output order indicated by the arrow 22 as described above. Figure 1 As described above, the output order of the decoding results of the points belonging to the nodes to be octree encoded is guaranteed as described above. In other words, the decoding results of points F, G, H, K, and L are output in the order of point F→point G→point H→point K→point L.

[0117] By comparison, since DCM is applied to points M, J, and N, points M, J, and N may not be Figure 3 With this in mind, the DCM order information is used to make the points output in the correct order.

[0118] Since the decoding result of each point is Figure 3 The decoding results are output from left to right in , so it is assumed that the correct output order of the decoding results of point M is 2, the correct output order of the decoding results of point J is 5, and the correct output order of the decoding results of point N is 6.

[0119] For example, the DCM sequence information may indicate the output order of the decoding results in order from the top (e.g., decoding order). In this case, the DCM sequence information indicates the output order of the decoding result of point M as "2", the output order of the decoding result of point J as "5", and the output order of the decoding result of point N as "6".

[0120] In addition, for example, the DCM sequence information may indicate the output order of the decoding results as a difference from the output order of the decoding results of the point to which the DCM is applied and output immediately before. In this case, the DCM sequence information indicates the output order of the decoding result of point M as "2" (a value obtained by subtracting the top "0" from the output order "2" of the decoding result of point M). In addition, the DCM sequence information indicates the output order of the decoding result of point J as "3" (a value obtained by subtracting the output order "2" of the decoding result of point M from the output order "5" of the decoding result of point J). Furthermore, the DCM sequence information indicates the output order of the decoding result of point N as "1" (a value obtained by subtracting the output order "5" of the decoding result of point J from the output order "6" of the decoding result of point N).

[0121] Therefore, compared with the case where the output order of the decoding results is indicated in order from the top (eg, decoding order), the encoding amount of the DCM order information can be reduced. In other words, a decrease in encoding efficiency can be suppressed.

[0122] In addition, for example, the DCM sequence information may indicate the output order of the decoding result as a difference from a predetermined reference order. For example, a grid may be set that divides the points into units consisting of a predetermined number of points, and the difference may be initialized (changed to "0") in such a grid unit. Figure 3 In the example shown in , the dot-dash line 23 indicates one grid unit. Therefore, in the case of this example, one grid unit is set for every five points. When the difference with the immediately preceding point to which the DCM is applied is obtained, once such a grid unit is exceeded, its difference is reset.

[0123] In this case, the DCM sequence information indicates the output order of the decoding result of point M as "2" (a value obtained by subtracting the top "0" from the output order "2" of the decoding result of point M). In addition, the DCM sequence information indicates the output order of the decoding result of point J as "3" (a value obtained by subtracting the output order "2" of the decoding result of point M from the output order "5" of the decoding result of point J). Furthermore, the DCM sequence information indicates the output order of the decoding result of point N as "1" (a value obtained by subtracting the grid unit "5" indicated by the dot-dash line 23 from the output order "6" of the decoding result of point N). Figure 3 In the example, "1" is used to indicate that the difference value has been initialized.

[0124] Therefore, compared with the case where the output order of the decoding results is indicated by the difference value from the immediately preceding point to which DCM is applied, the encoding amount of the DCM order information can be reduced. In other words, a decrease in encoding efficiency can be suppressed.

[0125] On the encoding side, such DCM sequence information is generated and transmitted to the decoding side. On the decoding side, the transmitted DCM sequence information is acquired, and the output order of the decoding results of the points to which DCM is applied is controlled based on the DCM sequence information. In other words, the decoding results of the points to which DCM is applied are output in the correct output order. Specifically, based on the DCM sequence information, the decoding results of the points to which DCM is applied are inserted into the position of the correct output order in the sequence of the decoding results of the octree encoded data arranged in the output order.

[0126] Therefore, since the need to perform sorting processing of points can be eliminated even when DCM is applied, an increase in the load of decoding processing of encoded data of a point cloud can be suppressed.

[0127] <Intermediate level control information>

[0128] The encoded data of the geometric data arranged in the tree structure as described above can be scalably decoded. In other words, the geometric data can be decoded at the resolution of the hierarchy (higher hierarchy than the lowest hierarchy) other than the lowest hierarchy.

[0129] However, the positions (output order) of the points to which DCM is applied as described above may vary depending on the hierarchy. Figure 4 As shown, it is assumed that the decoding result of each node (point) of the hierarchy surrounded by the solid line square 31 is output in the order indicated by the arrow 32.

[0130] In this case, the output order of points M, J (node ​​c), and N where DCM is applied is the same as Figure 3 The situation shown in is different.

[0131] Taking this into account, as in Figure 2 In "Method 1-5" described in the sixth row from the top row in the table shown in , control information (DCM sequence information) may also be generated at an intermediate level of the tree structure. For example, for a given point to which DCM is applied, control information (DCM sequence information) may be generated for all levels (or a part of the levels) from a node that is a processing object to be encoded by applying DCM to the given point.

[0132] In addition, such control information (DCM order information) may be signaled (transmitted from the encoding side to the decoding side). In addition, on the decoding side, the output order of the decoding results of the points to which the DCM is applied may be controlled based on such control information (DCM order information).

[0133] In other words, the control information (DCM order information) may be configured to include information on the output order of the decoding results in the middle level of the tree structure.

[0134] Even in the case of an intermediate layer, the control information may express the output order of the decoding results in a similar manner to the above-mentioned lowest layer. For example, the control information may indicate the output order of the decoding results of the points to which the DCM is applied in order from the top (e.g., decoding order). Figure 4 In the example shown in , the control information in this case indicates the output order of the decoding results of point M as "1", the output order of the decoding results of point J as "3", and the output order of the decoding results of point N as "4".

[0135] In addition, for example, the control information may indicate the output order of the decoding results of the points to which DCM is applied by a difference from the output order of the decoding results of the points to which DCM is applied and output immediately before. Figure 4 In the example shown in , the control information in this case indicates the output order of the decoding results of point M as "1" (a value obtained by subtracting the top "0" from the output order "1" of the decoding results of point M), indicates the output order of the decoding results of point J as "2" (a value obtained by subtracting the output order "1" of the decoding results of point M from the output order "3" of the decoding results of point J), and indicates the output order of the decoding results of point N as "1" (a value obtained by subtracting the output order "3" of the decoding results of point J from the output order "4" of the decoding results of point N).

[0136] Therefore, compared with the case where the output order of the decoding results is indicated by the order from the top (for example, the decoding order), the coding amount of the control information can be reduced. In other words, the decrease in coding efficiency can be suppressed.

[0137] In addition, for example, the control information of the point to which DCM is applied may indicate the output order of the decoding result by the difference from the predetermined reference order. For example, a grid that divides the points into units consisting of a predetermined number of points may be set, and the difference may be initialized (changed to "0") in such a grid unit. Figure 4 In the example shown in , the dot-dash line 33 indicates one grid unit. Therefore, in the case of this example, one grid unit is set for every three points. When the difference with the immediately preceding point to which the DCM is applied is obtained, once such a grid unit is exceeded, its difference is reset.

[0138] exist Figure 4 In the example shown in , the control information in this case indicates the output order of the decoding results of point M as "1" (a value obtained by subtracting the top "0" from the output order "1" of the decoding results of point M), indicates the output order of the decoding results of point J as "2" (a value obtained by subtracting the output order "1" of the decoding results of point M from the output order "3" of the decoding results of point J), and indicates the output order of the decoding results of point N as "1" (a value obtained by subtracting the grid unit "3" indicated by the dot-dash line 23 from the output order "4" of the decoding results of point N). Figure 4 In the example, "1" is used to indicate that the difference value has been initialized.

[0139] Therefore, compared with the case where the output order of the decoding results is indicated by the difference value from the immediately preceding point to which DCM is applied, the coding amount of the control information can be reduced. In other words, a decrease in coding efficiency can be suppressed.

[0140] like Figure 5 As shown in A in FIG. 1 , in the case of a conventional bit stream 40 of geometric data, the encoded data 41 of a point encoded by applying DCM and the encoded data 42 of a point subjected to octree encoding are configured as data segments that are different from each other and are not arranged in a processing order. In addition, there is no information indicating the processing order of each point. Furthermore, the points subjected to octree encoding are not arranged in a processing order. Therefore, the decoding results of the encoded data of each point must be sorted in the processing order of the attribute data (e.g., Morton order).

[0141] By comparison, Figure 5 As shown in B in FIG. 1 , as a result of applying the present technology described above, a bit stream 50 has coded data 51 of a point to which DCM coding is applied, coded data 52 of a point to which octree coding is performed, and DCM sequence information 53. The DCM sequence information 53 indicates the output order of the decoding result of each point to which DCM is applied. Therefore, on the decoding side, based on the DCM sequence information 53, the decoding result of each point to which DCM is applied is inserted into the correct position of the point group to which octree coding is performed.

[0142] Therefore, it is no longer necessary to sort the decoding results of the geometry data, and an increase in the load of the decoding process of the encoded data of the point cloud can be suppressed.

[0143] <Control Information Transmission Method>

[0144] The transmission method of the above control information is arbitrary. Figure 2 In "Method 1-6" described in the seventh row from the top row in the table shown in, the control information can be transmitted by a predetermined transmission method. For example, the control information can be included in the encoded data of the geometry data. Alternatively, for example, the control information can be transmitted as data different from the encoded data of the geometry data. In this case, the control information can be associated with the encoded data of the geometry data using identification information, etc. For example, the control information may include identification information of the geometry data of the corresponding data unit (e.g., slice).

[0145] <2. First Embodiment>

[0146] <Encoding device>

[0147] Next, a device to which the present technology described above in <1. Guarantee of output order of decoding results> is applied will be described. Figure 6 : is a block diagram showing an example of the configuration of an encoding device representing aspects of an information processing device to which the present technology is applied. Figure 6 The encoding device 100 shown in FIG. 1 is a device that encodes a point cloud (3D data). The encoding device 100 encodes a point cloud by applying the present technology described above in <1. Guarantee of output order of decoding results>.

[0148] It should be noted that Figure 6 shows the main processing parts, main data flows, etc., and Figure 6 These shown in the figure do not represent all processing units, all data flows, etc. In other words, in the encoding device 100, there may be Figure 6 The processing section shown as a block, and not in Figure 6 A process or data flow is illustrated by arrows in a diagram.

[0149] like Figure 6 As shown, the encoding device 100 includes a position information encoding unit 101, a position information decoding unit 102, a point cloud generating unit 103, an attribute information encoding unit 104 and a bit stream generating unit 105.

[0150] The position information encoding unit 101 encodes the geometric data (position information) of the point cloud (3D data) input to the encoding device 100. For example, the position information encoding unit 101 hierarchizes the geometric data and generates an octree, and encodes the octree. In addition, for example, the position information encoding unit 101 encodes the geometric data of sparse points by applying DCM.

[0151] In doing so, the position information encoding section 101 encodes the geometry data by applying the present technology described above in <1. Guarantee of output order of decoding results>. For example, the position information encoding section 101 arranges the geometry data into an octree and encodes the octree so that the decoding result of the geometry data is output in the processing order of the attribute data. The position information encoding section 101 provides the generated coded data of the geometry data to the position information decoding section 102 and the bitstream generating section 105.

[0152] The position information decoding section 102 acquires the encoded data of the geometric data supplied from the position information encoding section 101 and decodes the encoded data. The decoding method is arbitrary as long as the method corresponds to the encoding performed by the position information encoding section 101. For example, processing such as filtering or inverse quantization for denoising may be performed. The position information decoding section 102 supplies the generated geometric data (decoding result) to the point cloud generation section 103.

[0153] The point cloud generation unit 103 acquires the attribute data (attribute information) of the point cloud input to the encoding device 100 and the geometric data (decoding result) provided from the position information decoding unit 102. The point cloud generation unit 103 performs processing (recoloring processing) for matching the attribute data with the geometric data (decoding result). The point cloud generation unit 103 provides the attribute data associated with the geometric data (decoding result) to the attribute information encoding unit 104.

[0154] In addition, the point cloud generation unit 103 applies the present technology described above in <1. Guarantee of output order of decoding results> to perform processing for generating DCM order information indicating the output order of decoding results of points to which DCM is applied and encoding the DCM order information. The point cloud generation unit 103 provides the encoded data of the generated DCM order information to the bit stream generation unit 105.

[0155] The attribute information encoding unit 104 acquires the geometric data (decoding result) and the attribute data provided from the point cloud generation unit 103. The attribute information encoding unit 104 encodes the attribute data using the geometric data (decoding result) and generates the encoded data of the attribute data. The attribute information encoding unit 104 provides the generated encoded data of the attribute data to the bit stream generation unit 105.

[0156] The bitstream generation unit 105 acquires the encoded data of the geometric data provided from the position information encoding unit 101. In addition, the bitstream generation unit 105 acquires the encoded data of the attribute data provided from the attribute information encoding unit 104. In addition, the bitstream generation unit 105 acquires the encoded data of the DCM sequence information provided from the point cloud generation unit 103. The bitstream generation unit 105 generates a bitstream including these encoded data segments. The bitstream generation unit 105 outputs the generated bitstream to the outside of the encoding device 100.

[0157] By adopting such a configuration, since the encoding device 100 no longer needs to perform point sorting processing, it is possible to suppress an increase in the load of decoding processing of the encoded data of the point cloud. Therefore, the encoding device 100 can suppress an increase in the implementation cost for reliably implementing real-time decoding processing of the encoded data of the point cloud.

[0158] It should be noted that the processing units (position information encoding unit 101 to bit stream generating unit 105) of the encoding device 100 have arbitrary configurations respectively. For example, each processing unit may be composed of a logic circuit that implements the above-mentioned processing. In addition, each processing unit may have a CPU (central processing unit), a ROM (read-only memory), a RAM (random access memory), etc. and may implement the above-mentioned processing by executing a program using the CPU, ROM, RAM, etc. Needless to say, each processing unit may have two configurations, and may implement a part of the above-mentioned processing by a logic circuit, and another part of the above-mentioned processing by executing a program. Alternatively, the configurations of the respective processing units may be independent of each other, and for example, a part of the processing unit may implement a part of the above-mentioned processing by a logic circuit, another part of the processing unit may implement the above-mentioned processing by executing a program, and other processing units may implement the above-mentioned processing by using both a logic circuit and an executing program.

[0159] <Position Information Encoding Unit>

[0160] Figure 7 is a diagram showing the position information encoding unit 101 ( Figure 6 ) is a block diagram of a main configuration example. It should be noted that Figure 7 shows the main processing parts, main data flows, etc., and Figure 7 These shown in the figure do not represent all processing units, all data flows, etc. In other words, in the position information encoding unit 101, there may be Figure 7 The processing section shown as a block, and not in Figure 7 A process or data flow is illustrated by arrows in a diagram.

[0161] Figure 7 The position information encoding section 101 shown in FIG. 1 includes a bounding box setting section 111 , a voxel setting section 112 , a mode selection section 113 , an octree encoding section 114 , and a DCM encoding section 115 .

[0162] The bounding box setting unit 111 performs processing related to the setting of the bounding box. For example, the bounding box setting unit 111 obtains the geometric data of the point cloud data input to the encoding device 100. The bounding box setting unit 111 sets a bounding box for the geometric data. The bounding box is information for standardizing the geometric data as the encoding object. Voxelization is performed using the bounding box as a reference. The bounding box setting unit 111 provides the information about the bounding box together with the geometric data to the voxel setting unit 112.

[0163] The voxel setting section 112 performs processing related to the setting of voxels. For example, the voxel setting section 112 acquires information about the geometric data and the bounding box provided from the bounding box setting section 111. In addition, based on this information, the voxel setting section 112 sets voxels by dividing the bounding box set with respect to the geometric data. In other words, the voxel setting section 112 performs voxelization of the geometric data (quantization of the position of each point). The voxel setting section 112 provides the voxel data, which is the geometric data voxelized in this way, to the mode selection section 113.

[0164] The mode selection unit 113 performs processing related to the selection of the encoding method (mode). For example, the mode selection unit 113 acquires the voxel data provided from the voxel setting unit 112. In addition, the mode selection unit 113 performs the selection of the encoding method (mode) for each voxel (node ​​in the octree). In other words, the mode selection unit 113 selects whether to perform octree encoding or to perform encoding by applying DCM (also referred to as DCM encoding) for the voxel as the processing object.

[0165] For example, the mode selection unit 113 determines whether the voxel as the processing object is sparse. When it is determined based on the predetermined condition that the voxel is not sparse, the mode selection unit 113 selects octree encoding as the encoding method, and supplies the voxel data as the processing object to the octree encoding unit 114. On the other hand, when it is determined based on the predetermined condition that the voxel is sparse, the mode selection unit 113 selects DCM encoding as the encoding method, and supplies the voxel data as the processing object to the DCM encoding unit 115.

[0166] The octree encoding unit 114 performs processing related to encoding using an octree. For example, the octree encoding unit 114 acquires voxel data as a processing object provided from the mode selection unit 113. Using the voxel data, the octree encoding unit 114 generates octree data (ChildMask) of the processing object node. The octree encoding unit 114 encodes the octree data of the processing object node by a predetermined method to generate encoded data.

[0167] In doing so, the octree encoding unit 114 applies the present technology described above in <1. Guarantee of output order of decoding results> (for example, Figure 2 The encoding is performed in the "method 1" (which may include methods 1-1 to 1-3) in the octree encoding unit 114. In other words, the octree encoding unit 114 encodes the octree data as the processing object node so that the decoding result of the geometric data is output in the processing order of the attribute data. For example, the octree encoding unit 114 encodes the octree data as the processing object node so that the decoding result of the geometric data is output in the Morton order. For example, the octree encoding unit 114 performs encoding in such an order that the decoding result at each node of the octree is output in the processing order of the attribute data of the point cloud.

[0168] The octree encoding unit 114 supplies the encoded data (encoded data of the voxel data of the processing target node) generated by performing the encoding as described above to the position information decoding unit 102 and the bit stream generating unit 105 (both in Figure 6 ).

[0169] The DCM encoding unit 115 performs processing related to encoding using DCM. For example, the DCM encoding unit 115 obtains voxel data as a processing object provided from the mode selection unit 113. The DCM encoding unit 115 encodes the voxel data by applying DCM, and generates encoded data. For example, using the voxel data, the DCM encoding unit 115 encodes the relative distance from the processing object node to the leaf, and generates encoded data. The DCM encoding unit 115 provides the generated encoded data to the position information decoding unit 102 and the bit stream generation unit 105 (both in Figure 6 ).

[0170] By adopting such a configuration, the position information encoding unit 101 can perform octree encoding to ensure that the output order of the decoded results of the geometric data is a predetermined order. Therefore, since point sorting processing is no longer required, the encoding device 100 can suppress the increase in the load of the decoding processing of the encoded data of the point cloud.

[0171] It should be noted that each of the processing units (bounding box setting unit 111 to DCM encoding unit 115) of the position information encoding unit 101 has an arbitrary configuration. For example, each processing unit may be composed of a logic circuit that implements the above-mentioned processing. In addition, each processing unit may have a CPU, ROM, RAM, etc., and the above-mentioned processing may be implemented by executing a program using the CPU, ROM, RAM, etc. Needless to say, each processing unit may have two configurations, and a part of the above-mentioned processing may be implemented by a logic circuit, and another part of the above-mentioned processing may be implemented by executing a program. Alternatively, the configurations of the respective processing units may be independent of each other, and for example, a part of the processing unit may implement a part of the above-mentioned processing by a logic circuit, another part of the processing unit may implement the above-mentioned processing by executing a program, and other processing units may implement the above-mentioned processing by using both a logic circuit and an execution program.

[0172] <Point cloud generation section>

[0173] Figure 8 is a diagram showing the point cloud generation unit 103 ( Figure 6 ) is a block diagram of a main configuration example. It should be noted that Figure 8 shows the main processing parts, main data flows, etc., and Figure 8 The ones shown in the figure do not represent all processing units, all data flows, etc. In other words, in the point cloud generation unit 103, there may be Figure 8 The processing section shown as a block, and not in Figure 8 A process or data flow is illustrated by arrows in a diagram.

[0174] like Figure 8 As shown, the point cloud generating unit 103 includes a Morton code transforming unit 121 , a sorting unit 122 , a DCM sequence information generating unit 123 , a DCM sequence information encoding unit 124 and a recoloring processing unit 125 .

[0175] The Morton code conversion unit 121 converts the position information from the position information decoding unit 102 ( Figure 6 ) is converted into a Morton code. In other words, the Morton code conversion unit 121 maps a point in a three-dimensional space to one dimension using the Morton code. The Morton code conversion unit 121 provides the geometric data of each point to which the Morton code is added to the sorting unit 122.

[0176] The sorting unit 122 acquires the geometric data provided from the Morton code conversion unit 121 and performs sorting based on the value of the Morton code. In other words, the sorting unit 122 sorts the geometric data of each point in Morton order. The sorting unit 122 provides the geometric data of each point that has been sorted in Morton order to the DCM order information generation unit 123 and the recoloring processing unit 125.

[0177] The DCM order information generating section 123 specifies the point to which DCM is applied during encoding in the geometry data of each point which has been sorted in the Morton order, and generates DCM order information which is control information indicating the output order of the decoding result of the point.

[0178] For example, the DCM sequence information may include information indicating the output order of the decoding results of the points to which the DCM is applied in order from the top. In addition, the DCM sequence information may include information indicating the output order of the decoding results of the points to which the DCM is applied by a difference from the output order of the decoding results of the points to which the DCM is applied and output immediately before. In addition, the DCM sequence information may include information indicating the output order of the decoding results of the points to which the DCM is applied by a difference from a predetermined reference order.

[0179] It should be noted that the DCM order information generating unit 123 may be configured to generate DCM order information indicating the output order in the hierarchy of the point to which the DCM is applied even for the intermediate hierarchy of the octree. In other words, the DCM order information may be configured to include information related to the output order of the decoding results in the intermediate hierarchy of the octree. The DCM order information generating unit 123 provides the generated DCM order information to the DCM order information encoding unit 124.

[0180] The DCM sequence information encoding unit 124 performs processing related to the encoding of the DCM sequence information. For example, the DCM sequence information encoding unit 124 obtains the DCM sequence information provided by the DCM sequence information generating unit 123. The DCM sequence information encoding unit 124 encodes the obtained DCM sequence information and generates encoded data. The encoding method is arbitrary. The DCM sequence information encoding unit 124 provides the generated encoded data of the DCM sequence information to the bitstream generating unit 105 ( Figure 6 ).

[0181] The recoloring processing section 125 performs processing related to processing (recoloring processing) for matching attribute data with geometric data. For example, the recoloring processing section 125 acquires attribute data of the point cloud input to the encoding device 100. In addition, the recoloring processing section 125 acquires geometric data of each point that has been sorted in the Morton order and is provided from the sorting section 122.

[0182] The recoloring processing unit 125 performs processing (recoloring processing) for matching the acquired attribute data with the acquired geometric data and generates point cloud data. The recoloring processing unit 125 provides the generated point cloud data to the attribute information encoding unit 104 ( Figure 6 ).

[0183] By adopting the above configuration, the point cloud generation unit 103 can generate DCM sequence information and transmit the DCM sequence information to the decoding side. Due to the DCM sequence information, even the decoding results of the points that have been encoded by applying DCM can be output in an appropriate output order. Therefore, since it is no longer necessary to sort the points during decoding, the encoding device 100 can suppress the increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud.

[0184] It should be noted that the processing units (Morton code transformation unit 121 to recoloring processing unit 125) of the point cloud generation unit 103 each have an arbitrary configuration. For example, each processing unit may be composed of a logic circuit that implements the above-mentioned processing. In addition, each processing unit may have a CPU, ROM, RAM, etc., and the above-mentioned processing may be implemented by executing a program using the CPU, ROM, RAM, etc. Needless to say, each processing unit may have two configurations, and a part of the above-mentioned processing may be implemented by a logic circuit and the other part of the above-mentioned processing may be implemented by executing a program. Alternatively, the configurations of the respective processing units may be independent of each other, and for example, a part of the processing unit may implement a part of the above-mentioned processing by a logic circuit, another part of the processing unit may implement the above-mentioned processing by executing a program, and the other processing units may implement the above-mentioned processing by using both the logic circuit and the execution program.

[0185] <Encoding Process Flow>

[0186] Next, the processing to be performed by the encoding device 100 will be described. The encoding device 100 encodes the data of the point cloud by performing the encoding processing. Fig. 9 The flowchart shown in describes an example of the flow of the encoding process.

[0187] Once the encoding process is started, in step S101, the position information encoding section 101 of the encoding device 100 encodes the geometric data (position information) of the input point cloud and generates encoded data of the geometric data. In doing so, the position information encoding section 101 performs processing by applying the present technology described above in <1. Guarantee of output order of decoding results>. Details of the position information encoding process will be provided later.

[0188] In step S102 , the position information decoding section 102 decodes the encoded data of the geometric data generated in step S101 , and generates position information.

[0189] In step S103, the point cloud generation unit 103 performs a recoloring process using the attribute data (attribute information) of the input point cloud and the geometry data (decoding result) generated in step S102, and generates point cloud data by associating the attribute data with the geometry data. In addition, in doing so, the point cloud generation unit 103 applies the present technology described above in <1. Guarantee of output order of decoding results> to perform a process for generating DCM order information indicating the output order of decoding results of points to which DCM is applied and encoding the DCM order information. The details of the point cloud generation process will be described later.

[0190] In step S104 , the attribute information encoding section 104 performs attribute information encoding processing to encode the attribute data subjected to the recoloring processing in step S103 , and generates encoded data of the attribute data.

[0191] In step S105 , the bitstream generation section 105 generates a bitstream including the encoded data of the geometry data generated in step S101 , the encoded data of the DCM order information generated in step S103 , and the encoded data of the attribute data generated in step S104 , and outputs the generated bitstream.

[0192] When the process of step S105 ends, the encoding process ends.

[0193] By performing the processing of each step as described above, since the encoding device 100 no longer needs to perform point sorting processing during decoding, it is possible to suppress an increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, the encoding device 100 can suppress an increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud.

[0194] <Flow of Position Information Encoding Processing>

[0195] Next, we will refer to Fig.10 The flowchart in the description is in Fig. 9 An example of the flow of the position information encoding process performed in step S101 in FIG.

[0196] When the position information encoding process starts, in step S121 , the bounding box setting unit 111 sets a bounding box for the geometric data to be processed.

[0197] In step S122 , the voxel setting section 112 sets voxels based on the bounding box set in step S121 and quantizes the geometric data of each point.

[0198] In step S123 , the mode selection section 113 selects voxel data to be processed according to the Morton order from the voxels set in step S122 .

[0199] In step S124, the mode selection section 113 determines whether to apply DCM to the processing target voxel data. When the determination of sparseness is made based on the predetermined condition, the process proceeds to step S125.

[0200] In step S125, the DCM encoding unit 115 performs DCM encoding on the processing target voxel data. When the process of step S125 is completed, the process proceeds to step S127.

[0201] Alternatively, when it is determined in step S124 that DCM is not applied to the processing target voxel data, the process proceeds to step S126 .

[0202] In step S126, the octree encoding unit 114 applies the present technology described above in <1. Guarantee of output order of decoding results> to perform octree encoding on the processing target voxel data so that the output order of the decoding results is the Morton order. Once the processing of step S126 is completed, the processing proceeds to step S127.

[0203] In step S127, the mode selection unit 113 determines whether all voxel data segments have been processed. When there is voxel data to be processed, the process returns to step S123 and the subsequent process is repeated. In other words, each processing step from step S123 to step S127 is performed for each voxel data segment.

[0204] Subsequently, when it is determined in step S127 that all voxel data pieces have been processed, the position information encoding process ends and the process returns to step S128. Fig. 9 .

[0205] By performing the processing of each step as described above, since the encoding device 100 no longer needs to perform point sorting processing during decoding, it is possible to suppress an increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, the encoding device 100 can suppress an increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud.

[0206] <Flow of Point Cloud Generation Process>

[0207] Next, we will refer to Fig.11 The flowchart in the description is in Fig. 9 An example of the flow of point cloud generation processing performed in step S103 in FIG.

[0208] Once the point cloud generation process starts, in step S141, the Morton code conversion unit 121 ( Figure 8 ) transforms geometric data into Morton code.

[0209] In step S142 , the sorting section 122 sorts the geometry data in Morton order based on the Morton code added in step S121 .

[0210] In step S143, the DCM order information generating unit 123 specifies the point to which DCM is applied during encoding in the geometric data of each point that has been sorted in the Morton order, and generates DCM order information indicating the output order of the decoding result of the point. In doing so, the DCM order information generating unit 123 generates the DCM order information by applying the present technology described above in <1. Guarantee of output order of decoding results>.

[0211] In step S144 , the DCM order information encoding part 124 encodes the DCM order information generated in step S143 and generates encoded data.

[0212] In step S145 , the recoloring processing section 125 performs processing (recoloring processing) for matching the acquired attribute data with the acquired geometry data and generates point cloud data.

[0213] Once the processing of step S145 is completed, the point cloud generation processing is completed, and the processing returns to Fig. 9 .

[0214] By performing the processing of each step as described above, since the encoding device 100 no longer needs to perform point sorting processing during decoding, it is possible to suppress an increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, the encoding device 100 can suppress an increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud.

[0215] <3. Second Embodiment>

[0216] <Decoding device>

[0217] Next, another example of the device to which the present technology described above in <1. Guarantee of output order of decoding results> is applied will be described. Fig.12 : is a block diagram showing an example of the configuration of a decoding device representing aspects of an information processing device to which the present technology is applied. Fig.12 The decoding device 200 shown in FIG. 1 is a device that decodes the encoded data of the point cloud (3D data). The decoding device 200 decodes the encoded data of the point cloud by applying the present technology described above in <1. Guarantee of output order of decoding results>.

[0218] It should be noted that Fig.12 shows the main processing parts, main data flows, etc., and Fig.12 The ones shown in the figure do not represent all processing units, all data flows, etc. In other words, in the decoding device 200, there may be Fig.12 The processing section shown as a block, and not in Fig.12 A process or data flow is illustrated by arrows in a diagram.

[0219] like Fig.12 As shown, the decoding device 200 includes a coded data extraction unit 201 , a position information decoding unit 202 , an attribute information decoding unit 203 and a point cloud generation unit 204 .

[0220] The coded data extraction unit 201 acquires the bit stream input to the decoding device 200 and holds the bit stream. The coded data extraction unit 201 extracts coded data of geometric data (position information) and attribute data (attribute information) from the bit stream. In doing so, the coded data extraction unit 201 can extract coded data of all hierarchies from the bit stream. Alternatively, for example, coded data from the top layer to a layer specified by a user, an application, etc. (in other words, coded data of a part of the layer) can also be extracted from the bit stream.

[0221] The coded data extraction unit 201 supplies the extracted coded data of the geometry data to the position information decoding unit 202. The coded data extraction unit 201 supplies the extracted coded data of the attribute data to the attribute information decoding unit 203.

[0222] The position information decoding unit 202 acquires the encoded data of the geometric data provided from the encoded data extraction unit 201. The position information decoding unit 202 decodes the encoded data of the geometric data, and generates the geometric data (decoded result). In doing so, the position information decoding unit 202 decodes the geometric data by applying the present technology described above in <1. Guarantee of output order of decoding results>. In other words, the position information decoding unit 202 decodes the encoded data of the geometric data, and outputs its decoded result in the processing order of the attribute data of the point cloud.

[0223] The position information decoding unit 202 supplies the generated geometric data (decoding result) to the attribute information decoding unit 203 and the point cloud generating unit 204 .

[0224] The attribute information decoding unit 203 acquires the encoded data of the attribute data provided from the encoded data extraction unit 201. The attribute information decoding unit 203 acquires the geometric data (decoding result) provided from the position information decoding unit 202. Using the geometric data, the attribute information decoding unit 203 decodes the encoded data of the attribute data and generates the attribute data (decoding result). The attribute information decoding unit 203 provides the generated attribute data (decoding result) to the point cloud generation unit 204.

[0225] The point cloud generation unit 204 acquires the geometric data (decoding result) provided by the position information decoding unit 202. The point cloud generation unit 204 acquires the attribute data (decoding result) provided by the attribute information decoding unit 203. The point cloud generation unit 204 generates a point cloud (decoding result) using the geometric data (decoding result) and the attribute data (decoding result). The point cloud generation unit 204 outputs the data of the generated point cloud (decoding result) to the outside of the decoding device 200.

[0226] By adopting such a configuration, since the geometric data of the decoding result is output in the same order as the processing order of the attribute data, it is no longer necessary to perform point sorting processing. Therefore, the decoding device 200 can suppress the increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, it is possible to suppress the increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud.

[0227] It should be noted that the above-mentioned processing units (encoded data extraction unit 201 to point cloud generation unit 204) have arbitrary configurations, respectively. For example, each processing unit may be composed of a logic circuit that implements the above-mentioned processing. In addition, each processing unit may have a CPU, ROM, RAM, etc., and the above-mentioned processing may be implemented by executing a program using the CPU, ROM, RAM, etc. Needless to say, each processing unit may have two configurations, and a part of the above-mentioned processing may be implemented by a logic circuit, and the other part of the above-mentioned processing may be implemented by executing a program. Alternatively, the configurations of the respective processing units may be independent of each other, and for example, a part of the processing unit may implement a part of the above-mentioned processing by a logic circuit, another part of the processing unit may implement the above-mentioned processing by executing a program, and the other processing units may implement the above-mentioned processing by using both the logic circuit and the execution program.

[0228] <Position Information Decoding Unit>

[0229] Fig.13 is a diagram showing the position information decoding unit 202 ( Fig.12 ) is a block diagram of a main configuration example. It should be noted that Fig.13 shows the main processing parts, main data flows, etc., and Fig.13 These shown in the figure do not represent all processing units, all data flows, etc. In other words, in the position information decoding unit 202, there may be Fig.13 The processing section shown as a block, and not in Fig.13 A process or data flow is illustrated by arrows in a diagram.

[0230] Fig.13 The position information decoding unit 202 shown in FIG. 2 has a decoding unit 211 and a DCM inserting unit 212 .

[0231] The decoding section 211 performs processing related to decoding of geometry data. For example, the position information decoding section 202 acquires the encoded data of geometry data supplied from the encoded data extracting section 201. The decoding section 211 decodes the acquired encoded data and generates (restores) geometry data.

[0232] For example, the decoding unit 211 decodes the encoded data of the DCM sequence information included in the encoded data of the geometric data. In addition, for example, the decoding unit 211 decodes the encoded data of the octree data into the Morton order. In addition, for example, the decoding unit 211 decodes the encoded data that has been DCM-encoded. The decoding unit 211 provides the geometric data generated by decoding (which may include DCM sequence information, octree data, geometric data of points to which DCM is applied (also referred to as DCM data), etc.) to the DCM insertion unit 212.

[0233] The DCM inserting section 212 performs processing related to control of the output order of geometric data of points to which DCM is applied. For example, the DCM inserting section 212 acquires geometric data (which may include DCM order information, octree data, and DCM data) supplied from the decoding section 211.

[0234] The DCM inserting section 212 inserts the acquired DCM data into a position of the output order indicated by the DCM order information of the sequence of octree data sorted in the Morton order. In other words, the DCM inserting section 212 performs control so that the DCM data is output in the output order indicated by the DCM order information corresponding to the DCM data.

[0235] The DCM inserting unit 212 provides the octree data (geometric data sorted in Morton order) into which the DCM data is inserted to the point cloud generating unit 204 ( Fig.12 ).

[0236] By adopting such a configuration, since the geometric data of the decoding result is output in the same order as the processing order of the attribute data, it is no longer necessary to perform point sorting processing. Therefore, the decoding device 200 can suppress the increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, it is possible to suppress the increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud.

[0237] It should be noted that the above-mentioned processing units (decoding unit 211 and DCM insertion unit 212) have an arbitrary configuration. For example, each processing unit may be composed of a logic circuit that implements the above-mentioned processing. In addition, each processing unit may have a CPU, ROM, RAM, etc., and the above-mentioned processing may be implemented by executing a program using the CPU, ROM, RAM, etc. Needless to say, each processing unit may have two configurations, and a part of the above-mentioned processing may be implemented by a logic circuit, and the other part of the above-mentioned processing may be implemented by executing a program. Alternatively, the configurations of the respective processing units may be independent of each other, and for example, a part of the processing unit may implement a part of the above-mentioned processing by a logic circuit, another part of the processing unit may implement the above-mentioned processing by executing a program, and the other processing units may implement the above-mentioned processing by using both the logic circuit and the execution program.

[0238] <Decoding Process Flow>

[0239] Next, the processing to be performed by the decoding device 200 will be described. The decoding device 200 decodes the encoded data of the point cloud by performing the decoding processing. Fig.14 The flowchart shown in describes an example of the flow of the decoding process.

[0240] Once the decoding process starts, in step S201 , the coded data extraction unit 201 of the decoding device 200 acquires and holds a bit stream, and extracts coded data of geometric data (position information) and attribute data (attribute information) of a layer to be decoded from the bit stream.

[0241] In step S202, the position information decoding unit 202 decodes the encoded data of the geometric data extracted in step S201, and generates geometric data (decoding result). In doing so, the position information decoding unit 202 performs processing by applying the present technology described above in <1. Guarantee of output order of decoding results>. Details of the position information decoding processing will be provided later.

[0242] In step S203, the attribute information decoding section 203 decodes the encoded data of the attribute data extracted in step S201, and generates attribute data (decoding result).

[0243] In step S204 , the point cloud generation unit 204 generates point cloud data (decoded result) using the geometry data (decoded result) generated in step S202 and the attribute data (decoded result) generated in step S203 , and outputs the generated point cloud data (decoded result).

[0244] When the process of step S204 ends, the decoding process ends.

[0245] Since the processing of each step is performed in this way so that the geometric data of the decoding result is output in the same order as the processing order of the attribute data, it is no longer necessary to perform point sorting processing. Therefore, the decoding device 200 can suppress the increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, it is possible to suppress the increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud.

[0246] <Flow of Position Information Decoding Process>

[0247] Next, we will refer to Fig.15 The flowchart in the description is in Fig.14 An example of the flow of the position information decoding process performed in step S202 in .

[0248] When the position information decoding process starts, in step S221 , the decoding unit 211 of the position information decoding unit 202 decodes the DCM sequence information.

[0249] In step S222 , the decoding unit 211 decodes the encoded data of the octree data into the Morton order.

[0250] In step S223 , the decoding unit 211 decodes the encoded data of the DCM data.

[0251] In step S224, the DCM insertion section 212 inserts the DCM data decoded in step S223 into a position corresponding to the output order of the octree data, which is arranged in the Morton order and decoded in step S222, indicated by the DCM order information decoded in step S221. In other words, the DCM insertion section 212 performs control of the output order of the geometry data (decoding result) so that the DCM data is output in the output order indicated by the DCM order information. In doing so, the DCM insertion section 212 performs processing by applying the present technology described above in <1. Guarantee of output order of decoding results>.

[0252] Once the processing of step S224 is completed, the position information decoding processing is completed, and the processing returns to Fig.14 .

[0253] Since the processing of each step is performed in this way so that the geometric data of the decoding result is output in the same order as the processing order of the attribute data, it is no longer necessary to perform point sorting processing. Therefore, the decoding device 200 can suppress the increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, it is possible to suppress the increase in the implementation cost of real-time processing for reliably implementing the decoding of the encoded data of the point cloud.

[0254] <4. Appendix>

[0255] <Other output order>

[0256] Although a description has been given above of outputting the decoded results of the geometric data in the order in which the attribute data is processed, the output order of the decoded results of the geometric data only needs to be a known order and is not limited to the above example. As long as the output order is known, processing suitable for the output order can be performed in a subsequent stage and sorting of the points may become unnecessary.

[0257] For example, when the attribute data is not decoded, the output order of the decoded result of the geometry data may be an order other than the processing order of the attribute data.

[0258] like Fig.16 In the example shown in , when the geometric data of the partial area 302 (which is a three-dimensional area constituting a part of the bounding box 301 surrounding the object) is decoded, if the decoding result of the geometric data is output in Morton order, the geometric data included in the partial area 302 can be specified as a set of data. In other words, by simply specifying the first point (start number) and the last point (end number) as the decoding object, it is easy to decode only the geometric data within the partial area 302.

[0259] In this way, by applying the present technology described above in <1. Guarantee of output order of decoding results> to perform encoding or decoding, and causing the decoded results of geometric data to be output in a predetermined order, the need to perform point sorting processing can be eliminated. Therefore, in such a case, an increase in the load of decoding processing of the encoded data of the point cloud can be suppressed. Therefore, an increase in the implementation cost of real-time processing for reliably implementing decoding of the encoded data of the point cloud can be suppressed.

[0260] <Control Information>

[0261] Although the signal transmission of control information has been described above, alternatively, control information other than the above examples may be signaled. For example, control information (e.g., enabled_flag) for controlling whether to allow (or prohibit) the application of the above-mentioned present technology may be transmitted. In addition, for example, control information for specifying a range (e.g., an upper limit, a lower limit, or both an upper limit and a lower limit) in which the above-mentioned present technology is allowed (or prohibited) to be applied may be transmitted.

[0262] <Computer>

[0263] The above series of processes can be performed by hardware or by software. When a series of processes are to be performed by software, the program constituting the software is installed in a computer. In this case, examples of computers include computers built into dedicated hardware, and general-purpose personal computers that can perform various functions when various programs are installed therein, etc.

[0264] Fig.17 : is a block diagram showing a configuration example of hardware of a computer that executes the above-described series of processes according to a program.

[0265] exist Fig.17 In a computer 900 shown in , a CPU (Central Processing Unit) 901 , a ROM (Read Only Memory) 902 , and a RAM (Random Access Memory) 903 are connected to one another via a bus 904 .

[0266] An input / output interface 910 is also connected to the bus 904. An input section 911, an output section 912, a storage section 913, a communication section 914, and a drive 915 are connected to the input / output interface 910.

[0267] For example, the input unit 911 is composed of a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. For example, the output unit 912 is composed of a display, a speaker, an output terminal, etc. For example, the storage unit 913 is composed of a hard disk, a RAM disk, a non-volatile memory, etc. For example, the communication unit 914 is composed of a network interface, etc. The drive 915 drives a removable medium 921, which is a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc.

[0268] In the computer configured as described above, the series of processes described previously are performed when the CPU 901 loads the program stored in the storage section 913 onto the RAM 903 via the input / output interface 910 and the bus 904 and executes the program. Data required for the CPU 901 to perform various types of processing is also stored in the RAM 903 as appropriate.

[0269] For example, the program executed by the computer can be applied by recording it on the removable medium 921 as a package medium, etc. In this case, the program can be installed in the storage section 913 via the input / output interface 910 by mounting the removable medium 921 to the drive 915 .

[0270] In addition, the program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program can be received by the communication section 914 and installed in the storage section 913.

[0271] Alternatively, the program may be installed in the ROM 902 or the storage section 913 in advance.

[0272] <Target of application of this technology>

[0273] Although the above describes the case where the present technology is applied to the encoding and decoding of point cloud data, the present technology is not limited to these examples and can be applied to the encoding and decoding of 3D data of any standard. In other words, as long as there is no contradiction with the above-mentioned present technology, various types of processing such as encoding and decoding methods and specifications of various types of data such as 3D data and metadata are arbitrary. In addition, as long as there is no contradiction with the present technology, part of the above-mentioned processing and specifications can be omitted.

[0274] In addition, although the encoding device 100 and the decoding device 200 are described above as application examples of the present technology, the present technology can be applied to any configuration.

[0275] For example, the present technology can be applied to various electronic devices, including transmitters or receivers in satellite broadcasting, wired broadcasting such as cable television, broadcasting on the Internet, and broadcasting to terminals via cellular telecommunications (e.g., television receivers or mobile phones), and devices for recording images on media such as optical disks, magnetic disks, or flash memories and reproducing images from such storage media (e.g., hard disk recorders or camera devices).

[0276] In addition, for example, the present technology can also be implemented as a partial component of a device such as a processor of a system LSI (large scale integration) (e.g., a video processor), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a kit created by adding other functions to a unit (e.g., a video kit).

[0277] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology can be implemented as cloud computing in which processing is shared among multiple devices via a network and performed collaboratively by multiple devices. For example, the present technology can be implemented in a cloud service that provides services related to images (moving images) for any terminal such as a computer, AV (audio-visual) equipment, a mobile information processing terminal, or an IoT (Internet of Things) device.

[0278] It should be noted that in this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), and it is not important whether all the components exist in the same housing. Therefore, multiple devices housed in separate housings but connected to each other via a network and a single device that houses multiple modules in a single housing are both considered to be systems.

[0279] <Fields of application and use of this technology>

[0280] Systems, devices, processing units, etc. using this technology can be used in any field, such as transportation, medical care, security, agriculture, animal husbandry, mining, beauty, factories, home appliances, weather and nature monitoring. In addition, the use application is also arbitrary.

[0281] <Others>

[0282] In this specification, "flag" refers to information used to identify multiple states, and includes information that enables identification of three or more states in addition to information used when identifying the two states of true (1) and false (0). Therefore, for example, the value that a "flag" can take can be two values ​​of 1 / 0, or three or more values. In other words, the number of bits that constitute a "flag" is arbitrary, and can be one or more bits. In addition, since in addition to the form in which the identification information is included in the bit stream, the identification information (including the flag) is also conceived in the form in which the difference information of the identification information relative to some reference information is included in the bit stream, therefore, in this specification, "flag" and "identification information" include not only the information itself, but also the difference information relative to the reference information.

[0283] In addition, various types of information (such as metadata) related to the coded data (bitstream) can be transmitted or recorded in any form, as long as the information is associated with the coded data. In this case, the term "association" means, for example, when processing a data segment, enabling the use (linking) of another data segment. In other words, data segments associated with each other can be unified into one data segment, or can be processed as separate data segments. For example, information associated with the coded data (image) can be transmitted along a transmission path different from the coded data (image). In addition, for example, information associated with the coded data (image) can be recorded in a recording medium different from the coded data (image) (or a different recording area in the same recording medium). It should be noted that "association" can be applied to a part of the data, not all of the data. For example, an image and information corresponding to the image can be associated with each other in arbitrary units (e.g., multiple frames, a single frame, or a part of a frame).

[0284] In this specification, terms such as "synthesis", "multiplexing", "addition", "integration", "inclusion", "storage", "inclusion", "introduction" and "insertion" mean unifying multiple things into one, for example, unifying encoded data and metadata into one data segment, and represent a method of the above-mentioned "association".

[0285] In addition, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

[0286] For example, a configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configuration described above as a plurality of devices (or processing units) may be unified and configured as one device (or processing unit). In addition, a configuration other than the above configuration may be added to the configuration of each device (or each processing unit). In addition, as long as the configuration or operation of the entire system is substantially the same, a portion of the configuration of a given device (or processing unit) may be included in the configuration of another device (or processing unit).

[0287] In addition, for example, the above-mentioned program may be configured to be executed by any device. In such a case, the device only needs to be configured to include necessary functions (eg, functional blocks) and be able to obtain necessary information.

[0288] In addition, for example, each step of a flowchart may be performed by a single device or by multiple devices in a shared manner. In addition, when a single step includes multiple processing steps, the multiple processing steps may be performed by a single device or by multiple devices in a shared manner. In other words, the multiple processing steps included in a single step may be performed as a process of multiple steps. Conversely, a process described as multiple steps may also be performed together as a single step.

[0289] In addition, for example, in a program to be executed by a computer, the processing of the steps describing the program may be performed in time sequence or in parallel along the order described in this specification, or at a necessary timing such as when a call is made. In other words, the processing of each step may be performed in an order different from the above order as long as no contradiction occurs. Furthermore, the processing of the steps describing the program may be performed in parallel with the processing of other programs, or may be performed in combination with the processing of other programs.

[0290] In addition, for example, each of the multiple technologies related to the present technology can be performed independently of each other, as long as no contradiction occurs. Obviously, any number of multiple technologies can be performed in combination. For example, part or all of the present technology described in any embodiment can be performed in combination with part or all of the present technology described in other embodiments. In addition, part or all of any of the present technologies in the above-mentioned present technology can be performed in combination with other technologies not described above.

[0291] The present technology may also be configured as follows.

[0292] (1) An information processing device comprising:

[0293] An encoding unit, wherein the encoding unit is configured to:

[0294] encoding position information of a point cloud expressing an object having a three-dimensional shape as a collection of points in a sequence such that a decoding result of the encoded data of the position information is output in a processing sequence of attribute information of the point cloud, and

[0295] The encoded data is generated.

[0296] (2) The information processing device according to (1), wherein:

[0297] The encoding section is configured to encode the position information in such an order that the decoding result is output in a Morton order, and to generate the encoded data.

[0298] (3) The information processing device according to (1) or (2), wherein:

[0299] The encoding section is configured to encode the position information that has been arranged in a tree structure in an order in which the decoding result is output at each node of the tree structure in a processing order of the attribute information of the point cloud and generate the encoded data.

[0300] (4) The information processing device according to (3), wherein:

[0301] The tree structure is an octree.

[0302] (5) The information processing device according to (4), further comprising:

[0303] a DCM encoding section configured to encode position information of sparse points of the octree by applying DCM (Direct Coding Mode) and generate encoded data of the position information; and

[0304] A generating unit configured to generate control information related to an output order of a decoding result of the encoded data generated by the DCM encoding unit.

[0305] (6) The information processing device according to (5), wherein:

[0306] The control information includes information indicating an output order of the decoding results in order from the top.

[0307] (7) The information processing device according to (5), wherein:

[0308] The control information includes information indicating the output order of the decoding results by a difference value from the output order of the decoding results of the encoded data of the position information which has been encoded by applying the DCM and which is output immediately before.

[0309] (8) The information processing device according to (5), wherein:

[0310] The control information includes information indicating an output order of the decoding results by a difference from a predetermined reference order.

[0311] (9) The information processing device according to any one of (5) to (8), wherein:

[0312] The control information also includes information on an output order of the decoding results in an intermediate level of the tree structure.

[0313] (10) An information processing method comprising the following steps:

[0314] encoding position information of a point cloud expressing an object having a three-dimensional shape as a collection of points in a sequence such that a decoding result of the encoded data of the position information is output in a processing sequence of attribute information of the point cloud, and

[0315] The encoded data is generated.

[0316] (11) An information processing device comprising:

[0317] A decoding section configured to decode encoded data of position information of a point cloud expressing an object having a three-dimensional shape as a collection of points, and output a decoding result in a processing order of attribute information of the point cloud.

[0318] (12) The information processing device according to (11), wherein:

[0319] The decoding section is configured to output the decoding result in a Morton order.

[0320] (13) The information processing device according to (11) or (12), wherein:

[0321] The decoding section is configured to decode the encoded data of the position information that has been arranged in a tree structure, and output the decoding result in a processing order of the attribute information of the point cloud at each node of the tree structure.

[0322] (14) The information processing device according to (13), wherein:

[0323] The tree structure is an octree.

[0324] (15) The information processing device according to (14), further comprising:

[0325] An output control section is also included, the output control section being configured to cause a decoded result of the encoded data of the position information that has been encoded by applying DCM (Direct Coding Mode) to be output in an output order indicated by control information regarding an output order of the decoded result of the encoded data.

[0326] (16) The information processing device according to (15), wherein:

[0327] The control information includes information indicating an output order of the decoding results in order from the top.

[0328] (17) The information processing device according to (15), wherein:

[0329] The control information includes information indicating the output order of the decoding results by a difference value from the output order of the decoding results of the encoded data of the position information which has been encoded by applying the DCM and which is output immediately before.

[0330] (18) The information processing device according to (15), wherein:

[0331] The control information includes information indicating an output order of the decoding results by a difference from a predetermined reference order.

[0332] (19) The information processing device according to any one of (15) to (18), wherein:

[0333] The control information also includes information on an output order of the decoding results in an intermediate level of the tree structure.

[0334] (20) An information processing method comprising the following steps:

[0335] Encoded data of position information of a point cloud expressing an object having a three-dimensional shape as a set of points is decoded, and a decoding result is output in a processing order of attribute information of the point cloud.

[0336] Reference Mark List

[0337] 100 Encoding device

[0338] 101 Position information encoding unit

[0339] 102 Position information decoding unit

[0340] 103 Point cloud generation department

[0341] 104 attribute information encoding unit

[0342] 105 Bitstream Generation Unit

[0343] 111 Bounding Box Setting Section

[0344] 112 Voxel Setting Section

[0345] 113 Mode selection unit

[0346] 114 Octree Encoding Unit

[0347] 115 DCM Coding Department

[0348] 121 Morton code conversion unit

[0349] 122 Sorting Department

[0350] 123 DCM sequence information generation unit

[0351] 124 DCM sequence information encoding unit

[0352] 125 Recoloring Processing Department

[0353] 200 Decoding device

[0354] 201 Encoded data extraction unit

[0355] 202 Position information decoding unit

[0356] 203 Attribute information decoding unit

[0357] 204 Point Cloud Generation Department

[0358] 211 Decoding Department

[0359] 212 DCM Insertion< / dcm>

Claims

1. An information processing device, comprising: An encoding unit, wherein the encoding unit is configured to: encoding position information of a point cloud expressing an object having a three-dimensional shape as a collection of points in a sequence such that a decoding result of the encoded data of the position information is output in a processing sequence of attribute information of the point cloud, and generating the encoded data; wherein the encoding section is configured to encode the position information arranged in a tree structure and generate the encoded data in an order such that the decoding result is output at each node of the tree structure in the order of processing the attribute information of the point cloud, wherein the tree structure is an octree, and Wherein, the information processing device further includes: a direct encoding mode encoding section configured to encode position information of sparse points of the octree by applying a direct encoding mode and generate encoded data of the position information; and A generating section configured to generate control information related to an output order of a decoding result of the encoded data generated by the direct encoding mode encoding section, wherein the output order is an order of processing the attribute information.

2. The information processing device according to claim 1, wherein: The encoding section is configured to encode the position information in such an order that the decoding result is output in a Morton order, and to generate the encoded data.

3. The information processing device according to claim 1, wherein: The control information includes information indicating an output order of the decoding results in order from the top of the octree.

4. The information processing device according to claim 1, wherein: The control information includes information indicating the output order of the decoding results by a difference value from the output order of the decoding results of the encoded data of the position information which has been encoded by applying the direct encoding mode and which is output immediately before.

5. The information processing device according to claim 1, wherein: The control information includes information indicating an output order of the decoding results by a difference from a predetermined reference order.

6. The information processing device according to claim 1, wherein: The control information also includes information on an output order of the decoding results in an intermediate level of the tree structure.

7. An information processing method comprising the following steps: encoding position information of a point cloud expressing an object having a three-dimensional shape as a collection of points in a sequence such that a decoding result of the encoded data of the position information is output in a processing sequence of attribute information of the point cloud, and generating the encoded data; wherein the position information arranged in a tree structure is encoded and the encoded data is generated, the order causes the decoding result to be output at each node of the tree structure in the processing order of the attribute information of the point cloud, wherein the tree structure is an octree, and The information processing method further comprises the following steps: encoding the position information of the sparse points of the octree by applying a direct encoding mode, and generating encoded data of the position information; and Control information related to the output order of the generated decoding result of the encoded data is generated, wherein the output order is the processing order of the attribute information.

8. An information processing device, comprising: a decoding unit configured to decode encoded data of position information of a point cloud expressing an object having a three-dimensional shape as a collection of points, and output a decoding result in a processing order of attribute information of the point cloud, wherein the decoding unit is configured to decode the encoded data of the position information arranged in a tree structure, and output the decoding result in a processing order of the attribute information of the point cloud at each node of the tree structure, wherein the tree structure is an octree, and An output control unit, wherein the output control unit is configured to output the decoding results of the encoded data of the position information that has been encoded by applying a direct encoding mode in accordance with an output order indicated by control information related to the output order of the decoding results of the encoded data, wherein the output order is the processing order of the attribute information.

9. The information processing device according to claim 8, wherein: The decoding section is configured to output the decoding result in a Morton order.

10. The information processing device according to claim 8, wherein: The control information includes information indicating an output order of the decoding results in order from the top of the octree.

11. The information processing device according to claim 8, wherein: The control information includes information indicating the output order of the decoding results by a difference value from the output order of the decoding results of the encoded data of the position information which has been encoded by applying the direct encoding mode and which is output immediately before.

12. The information processing device according to claim 8, wherein: The control information includes information indicating an output order of the decoding results by a difference from a predetermined reference order.

13. The information processing device according to claim 8, wherein: The control information also includes information on an output order of the decoding results in an intermediate level of the tree structure.

14. An information processing method comprising the following steps: decoding encoded data of position information of a point cloud expressing an object having a three-dimensional shape as a collection of points, and outputting a decoding result in a processing order of attribute information of the point cloud, wherein the encoded data of the position information arranged in a tree structure is decoded, and the decoding result is outputted in a processing order of attribute information of the point cloud at each node of the tree structure, wherein the tree structure is an octree, and The decoded results of the encoded data of the position information which has been encoded by applying the direct encoding mode are output in an output order indicated by control information related to an output order of the decoded results of the encoded data, wherein the output order is a processing order of the attribute information.

15. A computer-readable storage medium having a computer-executable program stored thereon, which, when executed by a processor, causes the processor to execute the information processing method according to claim 7 or 14.

16. A computer program product comprising a computer executable program, which, when executed by a processor, causes the processor to execute the information processing method according to claim 7 or the information processing method according to claim 14.

Images