Information processing device and method
By using Octree encoding and DCM to output point cloud data in a predetermined order, the decoding process is optimized, reducing processing load and enabling real-time decoding.
Patent Information
- Application Number
- JP2021532790
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-07-17
- Filing Date
- 2020-07-06
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-07-06
AI Technical Summary
The sorting process for encoding point cloud data increases the load of decoding processing, making real-time decoding difficult and increasing implementation costs.
An information processing device and method that uses Octree encoding and Direct Coding Mode (DCM) to ensure the decoding results of point cloud data are output in a predetermined order, eliminating the need for point rearrangement processing.
This approach reduces the load of decoding processing, enabling real-time decoding of point cloud data by ensuring the decoding results are output in a predetermined order, thus reducing implementation costs.
Smart Images

Figure 0007586078000001 
Figure 0007586078000002 
Figure 0007586078000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that are capable of suppressing an increase in the load of decoding processing of encoded data of a point cloud. [Background technology]
[0002] Conventionally, a method for encoding 3D data representing a three-dimensional structure such as a point cloud has been considered (see, for example, Non-Patent Document 1). Point cloud data is composed of geometry data (also called position information) and attribute data (also called attribute information) of each point. Therefore, encoding of a point cloud is performed for each of the geometry data and attribute data.
[0003] Various methods have been proposed as methods for encoding attribute data. For example, a technique called Lifting has been proposed (see, for example, Non-Patent Document 2). A method has also been proposed that enables attribute data to be decoded in a scalable manner (see, for example, Non-Patent Document 3). Also, although not using Lifting, a method has been devised that enables decoded attribute data to be referenced within the LoD (see, for example, Non-Patent Document 4).
[0004] In either case, the attribute data is encoded using the positional relationships between points, assuming that the geometry data is known, including degradation due to compression. More specifically, the attribute data is encoded using the correlation between points that are spatially close to each other. In order to utilize such correlation between points, points in three-dimensional space are mapped into one dimension using a Morton code, and sorted by the value of the Morton code. In other words, each point is sorted in Morton order. This ensures that adjacent points in the mapped one dimension are spatially close to each other. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] R. Mekuria, Student Member IEEE, K. Blom, P. Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video",tcsvt_paper_submitted_february.pdf [Non-Patent Document 2] Khaled Mammou, Alexis Tourapis, Jungsun Kim, Fabrice Robinet, Valery Valentin, Yeping Su, "Lifting Scheme for Lossy Attribute Encoding in TMC1", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42640, April 2018, San Diego, US [Non-Patent Document 3] Ohji Nakagami, Satoru Kuma, "[G-PCC] Spatial scalability support for G-PCC", ISO / IEC JTC1 / SC29 / WG11 MPEG2019 / m47352, March 2019, Geneva, CH [Non-Patent Document 4] Toshiyasu Sugio, "[G-PCC] Reference structure modification on attribute predicting transform in TMC13", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m46107, January 2019, Marrakech, MA Summary of the Invention [Problem to be solved by the invention]
[0006] However, this sorting process may increase the load of the decoding process of the encoded data of the point cloud.
[0007] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress an increase in the load of decoding processing of encoded data of a point cloud. [Means for solving the problem]
[0008] According to one aspect of the present technology, there is provided an information processing apparatus for processing position information of a point cloud that represents a three-dimensional object as a set of points, Octree Encoding in such an order that the decoded results are output in the processing order of the attribute information of the point cloud; 1. Generate the encoded data an Octree encoding unit, a DCM encoding unit that encodes position information of the sparsely populated portion of the point cloud using a Direct Coding Mode (DCM) encoding method to generate second encoded data, and a generating unit that generates control information related to an output order of the decoded result of the generated second encoded data. The information processing device includes:
[0009] According to one aspect of the present technology, there is provided an information processing method for processing position information of a point cloud that represents a three-dimensional object as a set of points, Octree Encoding in such an order that the decoded results are output in the processing order of the attribute information of the point cloud; 1. Generate the encoded data encoding position information of the sparsely-spaced portion of the point cloud using a direct coding mode (DCM) to generate second encoded data; and generating control information related to an output order of the decoded result of the generated second encoded data. It is an information processing method.
[0010] According to another aspect of the present technology, there is provided an information processing apparatus for processing position information of a point cloud that represents an object of a three-dimensional shape as a set of points. The first one is encoded as an octree. Decoding the encoded data; The first encoded data The decoded results are output in the order of processing of the attribute information of the point cloud. and further decoding second encoded data in which position information of the portion of the point cloud where the points are sparsely distributed is encoded using a Direct Coding Mode (DCM). Decoding section an output control unit that outputs a result of decoding the second encoded data in an output order indicated by control information regarding an output order of the result of decoding the second encoded data; The information processing device includes:
[0011] Another aspect of the information processing method of the present technology is to process position information of a point cloud that represents a three-dimensional object as a set of points. The first one is encoded as an octree. Decoding the encoded data; The first encoded dataThe decoded results are output in the order of processing of the attribute information of the point cloud. and further decoding second encoded data in which the position information of the point cloud is encoded using a Direct Coding Mode (DCM). do and outputting the decoded result of the second encoded data in an output order indicated by control information related to an output order of the decoded result of the second encoded data. It is an information processing method.
[0012] In the information processing device and method according to one aspect of the present technology, position information of a point cloud that represents an object of a three-dimensional shape as a set of points is Convert it into an octree, Encoding is done in such a way that the decoded results are output in the order of processing the attribute information of the point cloud. And the first Encoded Data The method further comprises: generating a point cloud of sparsely-spaced points; encoding position information of the point cloud in a direct coding mode (DCM) to generate second encoded data; and generating control information related to an output order of the decoded result of the generated second encoded data. .
[0013] In the information processing device and method according to another aspect of the present technology, position information of a point cloud that represents an object of a three-dimensional shape as a set of points is The first one is encoded as an octree. Encoded Data and decode the first encoded data Decryption result of Output point cloud attribute information in processing order Furthermore, the method includes decoding second encoded data in which position information of sparsely-spaced parts of the point cloud is encoded using Direct Coding Mode (DCM), and outputting the decoded results of the second encoded data in an output order indicated by control information relating to the output order of the decoded results of the second encoded data. [Brief description of the drawings]
[0014] [Figure 1] 11A and 11B are diagrams illustrating an example of a processing order of geometry data. [Diagram 2] FIG. 2 is a diagram illustrating a geometry data encoding method and a decoding method. [Diagram 3] FIG. 13 is a diagram for explaining a technique when DCM is applied. [Figure 4] FIG. 13 is a diagram for explaining a technique when DCM is applied. [Diagram 5] FIG. 13 is a diagram for explaining a technique when DCM is applied. [Figure 6] FIG. 2 is a block diagram showing an example of the main configuration of an encoding device. [Figure 7] 4 is a block diagram showing an example of the main configuration of a position information encoding unit. FIG. [Figure 8] 11 is a block diagram showing an example of the main configuration of a point cloud generating unit. FIG. [Figure 9] 11 is a flowchart illustrating an example of the flow of an encoding process. [Figure 10]13 is a flowchart illustrating an example of the flow of a position information encoding process. [Figure 11] 13 is a flowchart illustrating an example of the flow of a point cloud generation process. [Figure 12] FIG. 2 is a block diagram showing an example of the main configuration of a decoding device. [Figure 13] 13 is a block diagram showing an example of the main configuration of a position information decoding unit. FIG. [Figure 14] 13 is a flowchart illustrating an example of the flow of a decoding process. [Figure 15] 13 is a flowchart illustrating an example of the flow of a position information decoding process. [Figure 16] FIG. 11 is a diagram illustrating an example of decoding of a local region. [Figure 17] FIG. 1 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Guarantee of output order of decryption results 2. First embodiment (encoding device) 3. Second embodiment (decoding device) 4. Notes
[0016] <1. Guaranteeing the output order of decryption results> <References supporting technical content and technical terms> The scope of disclosure of the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent documents that were publicly known at the time of filing the application.
[0017] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: (mentioned above) Non-patent document 5: Sebastien Lasserre, David Flynn, "[PCC] Inference of a mode using point location direct coding in TMC3", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42239, January 2018, Gwangju, Korea
[0018] In other words, the contents described in these non-patent documents also serve as a basis for determining support requirements.
[0019] <Point cloud> Previously, there was 3D data such as point clouds, which represent three-dimensional structures based on point group position information and attribute information, and meshes, which are composed of vertices, edges, and faces and define three-dimensional shapes using polygonal representations.
[0020] For example, in the case of a point cloud, a three-dimensional structure (an object having a three-dimensional shape) is represented as a collection of many points (a point cloud). In other words, the data of the point cloud (also referred to as point cloud data) is composed of geometry data (also referred to as position information) and attribute data (also referred to as attribute information) of each point of the point cloud. The attribute data can include any information. For example, color information, reflectance information, normal information, etc. may be included in the attribute data. Therefore, the data structure is relatively simple, and any three-dimensional structure can be represented with sufficient accuracy by using a sufficiently large number of points.
[0021] <Quantization of position information using voxels> Since such point cloud data has a relatively large amount of data, an encoding method using voxels was devised to compress the amount of data by encoding, etc. A voxel is a three-dimensional region for quantizing geometry data (position information).
[0022] That is, the three-dimensional area containing the point cloud is divided into small three-dimensional areas called voxels, and each voxel indicates whether it contains a point or not. In this way, the position of each point is quantized in voxel units. Therefore, by converting point cloud data into such voxel data (also called voxel data), it is possible to suppress an increase in the amount of information (typically, to reduce the amount of information).
[0023] <octree> Furthermore, for geometry data, it was considered to construct an octree using such voxel data. An octree is a tree-structured version of voxel data. The value of each bit in the lowest node of this octree indicates whether or not each voxel has a point. For example, a value of "1" indicates a voxel that contains a point, and a value of "0" indicates a voxel that does not contain a point. In an octree, one node corresponds to eight voxels. In other words, each node in the octree is made up of eight bits of data, and these eight bits indicate whether or not each of the eight voxels has a point.
[0024] The upper nodes of the Octree indicate whether or not there is a point in the area that combines the eight voxels corresponding to the lower nodes belonging to that node. In other words, the upper nodes are generated by combining the information of the voxels of the lower nodes. Note that a node with a value of "0", that is, a node in which none of the corresponding eight voxels contains a point, is deleted.
[0025] In this way, a tree structure (Octree) consisting of nodes whose values are not "0" is constructed. In other words, the Octree can indicate the presence or absence of voxel points at each resolution. By encoding the Octree, the position information can be decoded from the highest resolution (top layer) to the desired layer (resolution) to restore the point cloud data at that resolution. In other words, it is possible to easily decode at any resolution without decoding information at unnecessary layers (resolutions). In other words, it is possible to achieve scalability of voxels (resolutions).
[0026] In addition, by omitting nodes with a value of "0" as described above, it is possible to reduce the resolution of voxels in areas where no points exist, thereby further suppressing the increase in the amount of information (typically reducing the amount of information).
[0027] <lifting> On the other hand, when encoding attribute data (attribute information), the geometry data (position information) is assumed to be known, including degradation due to encoding, and encoding is performed using the positional relationship between points. As a method for encoding such attribute data, a method using a transformation called RAHT (Region Adaptive Hierarchical Transform) or Lifting as described in Non-Patent Document 2 has been considered. By applying these technologies, it is also possible to hierarchically organize attribute data, like the Octree of geometry data.
[0028] For example, in the case of Lifting described in Non-Patent Document 2, attribute data of each point is encoded as a difference value with a predicted value derived using attribute data of other points. At that time, each point is hierarchically organized, and the difference value is derived according to the hierarchical structure.
[0029] That is, for the attribute data of each point, each point is classified into a prediction point and a reference point, a prediction value of the attribute data of the prediction point is derived using the attribute data of the reference point, and a difference value between the attribute data of the prediction point and the prediction value is derived. By recursively repeating such processing for the reference points, the attribute data of each point is hierarchically organized.
[0030] However, this hierarchical structure is generated independently of the hierarchical structure of the geometry data (e.g., Octree), and does not basically correspond to the hierarchical structure of the geometry data. In order to restore the point cloud data, it is necessary to make the geometry data correspond to the attribute data, and for that purpose, it is necessary to decode the geometry data and the attribute data up to the highest resolution (i.e., the lowest layer). In other words, the method to which Lifting described in Non-Patent Document 2 is applied does not support resolution scalable decoding.
[0031] <Hierarchical structure for scalable decryption> In contrast, the hierarchical structure described in Non-Patent Document 3 supports resolution scalable decoding. In the method described in Non-Patent Document 3, attribute data is hierarchical structured to match the hierarchical structure of the octree of the geometry data. In other words, when a point exists in an area corresponding to a voxel of the geometry data (attribute data corresponding to the point exists), a reference point and a predicted point are selected so that a point also exists in a voxel one level above the voxel (attribute data corresponding to the point exists). In other words, attribute information is hierarchical structured according to the hierarchical structure of the octree of the geometry data.
[0032] By associating the hierarchical structure of the attribute data with the hierarchical structure of the geometry data, it is possible to easily restore point cloud data of a desired resolution without decoding down to the lowest layer. In this way, the method applying the technology described in Non-Patent Document 3 supports resolution scalable decoding.
[0033] <Sorting process> In either case, the encoding and decoding of attribute data is performed by utilizing the correlation between points that are spatially close to each other. To utilize this correlation between points, points in 3D space are mapped to one dimension using a Morton code, and sorted by the value of the Morton code. In other words, a rearrangement process is performed on the geometry data, and each point is arranged in Morton order. This ensures that adjacent points in the one-dimensional mapping are spatially close to each other.
[0034] For example, in the octree of the geometry data shown in Fig. 1, nodes b to d belong to node a, points F to H belong to node b, point J belongs to node c, and points K and L belong to node d. Nodes a, b, and d are Octree-coded, and node c is coded by applying DCM (Direct Coding Mode).
[0035] By decoding nodes a through d enclosed in a dotted circle 11 of the coded data of an octree having such a configuration, points F through L enclosed in a solid square 12 are output.
[0036] In this case, the processing starts from node a, followed by nodes b, c, and d. However, the processing order of nodes b, c, and d is not specified, and they can be performed in parallel. Also, since node c is encoded by applying DCM, point J is generally output earlier than other points.
[0037] In addition, by processing node b, points F to H are output as decoded results, but the order in which they are output is not specified. Similarly, by processing node d, points K and L are output as decoded results, but the order in which they are output is not specified.
[0038] Therefore, the output order of each point is as follows: Note that in the following, the output order of points in parentheses is random. J,((F,G,H),(K,L))
[0039] That is, the output order of points F to H is random, the output order of points K and L is random, and the output order of points F to H and points K and L is random.
[0040] In this way, the output order of the geometry data of each point has a certain degree of freedom, and therefore there is a risk that it may not match the processing order of the attribute data. In other words, it is not guaranteed that the geometry data of each point is output in the processing order of the attribute data. Therefore, in order to process the attribute data in the above-mentioned processing order, it is necessary to perform a point rearrangement process using the geometry data as described above.
[0041] However, this sorting process has a high computational cost, i.e., a large processing load. In other words, the amount of processing and the processing time of this sorting process are not small. Therefore, by performing this sorting process, the load of the decoding process of the encoded data of the point cloud may increase. For example, the increase in computational cost due to this sorting process may make it difficult to process the decoding of the encoded data of the point cloud in real time. Therefore, in order to reliably realize the real-time processing of the decoding of the encoded data of the point cloud, the implementation cost may increase.
[0042] <Guaranteeing the output order of decryption results> Therefore, as in "Method 1" in the top row of the table shown in FIG. 2, the output order of the decoded results of the geometry data is set to a predetermined order. For example, the geometry data is encoded so that the output order of the decoded results of the geometry data is guaranteed to be a predetermined order. In this way, the rearrangement process of the points is not required, so that an increase in the load of the decoding process of the encoded data of the point cloud can be suppressed. Therefore, an increase in the implementation cost for reliably realizing the real-time processing of the decoding of the encoded data of the point cloud can be suppressed.
[0043] Also, for example, the encoded data of the geometry data is decoded so that the decoded results are output in a predetermined order. In this way, the rearrangement of points is not required, so that the increase in the load of the decoding process of the encoded data of the point cloud can be suppressed. Therefore, the increase in the implementation cost for reliably realizing the real-time processing of the decoding of the encoded data of the point cloud can be suppressed.
[0044] For example, in the case of the octree of geometry data shown in Fig. 1, it is guaranteed that in node b, the decoded results of points F to H are output in the order of point F → point G → point H. Also, in node d, it is guaranteed that the decoded results of points K and L are output in the order of point K → point L. Furthermore, it is guaranteed that the decoded results of points (points F to H) belonging to each are output in the order of node b → node d.
[0045] By doing this, the output order of each point will be as follows: J, F, G, H, K, L
[0046] In other words, the output order is fixed and known. Therefore, the rearrangement of points is not required, and an increase in the load of the decoding process of the encoded data of the point cloud can be suppressed.
[0047] This predetermined order may be any order that is known when decoding the encoded data of the geometry data. For example, the predetermined order may be the Morton order, as in "Method 1-1" in the second row from the top of the table shown in Fig. 2. Outputting in the Morton order ensures that adjacent points in the output order are spatially close to each other.
[0048] Also, for example, this predetermined order may be the processing order of the attribute data, as in "Method 1-2" described in the third row from the top of the table shown in Fig. 2. In other words, the output order of the decoded results of the geometry data may be the same as the processing order of the attribute data.
[0049] For example, the geometry data (position information) of a point cloud, which represents a three-dimensional object as a collection of points, may be encoded in an order such that the decoded result of the encoded data of the geometry data is output in the processing order of the attribute data (attribute information) of the point cloud, and the encoded data may be generated.
[0050] For example, an information processing device may include an encoding unit that encodes geometry data (position information) of a point cloud that represents a three-dimensional object as a collection of points in such an order that the decoded result of the encoded data of the geometry data is output in the processing order of the attribute data of the point cloud, and generates the encoded data.
[0051] By encoding in this way, it is guaranteed that the decoded geometry data is output in the same order as the attribute data is processed during decoding. Therefore, the rearrangement of points is not required, so that the load of the decoding process of the encoded data of the point cloud can be suppressed. Therefore, the increase in the implementation cost for reliably realizing the real-time processing of the decoding of the encoded data of the point cloud can be suppressed.
[0052] The geometry data may be encoded in any manner. For example, when CABAC (Context-based Adaptive Binary Arithmetic Code) is used to encode the geometry data, the encoded data is decoded in the same order as the encoding order because a context is used. Therefore, in this case, by encoding the geometry data in the processing order of the attribute data, it is guaranteed that the decoded geometry data is output in the same order as the processing order of the attribute data.
[0053] Also, for example, encoded data of position information of a point cloud that represents a three-dimensional object as a collection of points may be decoded, and the decoded results may be output in the processing order of the attribute information of the point cloud.
[0054] For example, an information processing device may include a decoding unit that decodes encoded data of position information of a point cloud that represents a three-dimensional object as a collection of points, and outputs the decoded results in the processing order of the attribute information of the point cloud.
[0055] By decoding in this way, the geometry data of the decoded result is output in the same order as the attribute data processing order, so there is no need to rearrange the points. Therefore, it is possible to suppress an increase in the load of the decoding process of the encoded data of the point cloud. Therefore, it is possible to suppress an increase in the implementation cost for reliably realizing the real-time processing of the decoding of the encoded data of the point cloud.
[0056] The method of decoding the geometry data is arbitrary as long as it corresponds to the encoding method. For example, when the geometry data is encoded by applying CABAC, the context is used in decoding the encoded data, so that the encoded data of the geometry data can be decoded according to the order of the data (i.e., the encoding order of the geometry data). By decoding in this way, it is guaranteed that the decoded geometry data is output in the same order as the processing order of the attribute data.
[0057] The output order of the decoded results (i.e., the processing order of the attribute data) may be any order. For example, the decoded results may be output in Morton order (i.e., the attribute data may be processed in Morton order). Outputting the results in Morton order ensures that adjacent points in the output order are spatially close to each other.
[0058] The geometry data may be tree-structured. Furthermore, as in "Method 1-3" described in the fourth row from the top of the table shown in FIG. 2, it may be ensured that the output order of the decoded results at each node of the tree structure is a predetermined order (for example, the processing order of attribute data, Morton's order, etc.). For example, the tree-structured geometry data may be encoded in such an order that the decoded results at each node of the tree structure are output in the processing order of the attribute data of the point cloud. Also, for example, the encoded data of the tree-structured geometry data may be decoded, and the decoded results at each node of the tree structure may be output in the processing order of the attribute data of the point cloud.
[0059] By doing this, even when encoding and decoding tree-structured geometry data, it is possible to eliminate the need for point rearrangement processing, and it is possible to suppress an increase in the load of the decoding processing of the encoded data of the point cloud.
[0060] This tree structure may be of any type, for example, an Octree, or a KD-tree.
[0061] <dcm> In addition, for the encoding method of tree-structured geometry data, not only the method using the tree structure (for example, Octree encoding using Octree) but also the method applying DCM (Direct Coding Mode) may be used.
[0062] For example, in the Octree conversion of voxel data, when the node to be processed satisfies a predetermined condition and is determined to be sparse, this DCM is applied, and the relative distances (for each of the xyz directions) from the node to be processed to each leaf (point) directly or indirectly belonging to the node to be processed are obtained and encoded.
[0063] Note that the "node directly belonging" indicates a node hanging under the target node in the tree structure. For example, a node directly belonging to the node to be processed indicates a node belonging to the node to be processed and being one level lower than the node to be processed (so-called child node). Also, the "node indirectly belonging" indicates a node hanging under the target node via another node in the tree structure. For example, a node indirectly belonging to the node to be processed indicates a node belonging to the node to be processed via another node and being two or more levels lower than the node to be processed (for example, grandchild node).
[0064] That is, by applying DCM, the encoding and decoding of the intermediate-level nodes between the node to be processed and each leaf directly or indirectly belonging to the node to be processed can be omitted. In other words, when encoding and decoding the node to be processed, each leaf directly or indirectly belonging to the node to be processed can be encoded and decoded. Therefore, an increase in the load of encoding and decoding can be suppressed.
[0065] <Guarantee of Output Order Corresponding to DCM> Even when such a DCM is applied, it may be possible to guarantee that the output order of the decoded results of the geometry data is a predetermined order. However, the processing order of the leaf (point) to which the DCM is applied depends on the processing order of the node to be processed, and is difficult to control, and may differ from the case of Octree encoding. In other words, it is difficult to control the output order of the decoded results of the point to which the DCM is applied.
[0066] Therefore, as in "Method 1-4" described in the fifth row from the top of the table shown in Figure 2, control information regarding the output order of the decoding results for the points to which DCM is applied may be signaled (transmitted from the encoding side to the decoding side).
[0067] For example, on the encoding side, geometry data of sparse points may be encoded by applying DCM, encoded data of the geometry data may be generated, and control information (also referred to as DCM order information) regarding the output order of the decoding results of the encoded data may be generated.
[0068] Also, for example, on the decoding side, the decoded results of the coded data of the geometry data coded by applying the DCM may be output in the output order indicated by the DCM order information.
[0069] By doing this, it is guaranteed that the points to which DCM is applied are output in the appropriate output order, so that even when DCM is applied, the need for point rearrangement can be eliminated, and an increase in the load of the decoding process of the encoded data of the point cloud can be suppressed.
[0070] The DCM order information may be any information that can indicate the output order of the decoded results of the points to which DCM has been applied. For example, the DCM order information may include information indicating the output order of the decoded results of the points to which DCM has been applied in order from the beginning. The DCM order information may also include information indicating the output order of the decoded results of the points to which DCM has been applied in terms of a difference value from the output order of the decoded results of the point to which DCM has been applied that is output immediately before. Furthermore, the DCM order information may include information indicating the output order of the decoded results of the points to which DCM has been applied in terms of a difference value from a predetermined reference order.
[0071] For example, in the Octree shown in Fig. 3, nodes a to d and points F to L are configured in the same manner as in Fig. 1. Node a belongs to node e. Points M and L belonging to this node e are encoded and decoded by applying DCM.
[0072] In such a case, it is guaranteed that points F to N surrounded by solid-line rectangle 21 are output in the output order indicated by arrow 22 as described above. The output order of the decoded results of points belonging to nodes that are Octree-encoded as described with reference to Fig. 1 is guaranteed as described above. In other words, the decoded results of points F, G, H, K, and L are output in the order of point F → point G → point H → point K → point L.
[0073] On the other hand, points M, J, and N are subject to DCM, and therefore are not necessarily processed in the order shown in Fig. 3. Therefore, DCM order information is used to output these points in the correct order.
[0074] In Figure 3, the decoded results of each point are output in order from left to right, so the correct output order of the decoded result of point M is 2, the correct output order of the decoded result of point J is 5, and the correct output order of the decoded result of point N is 6.
[0075] For example, the DCM order information may indicate the output order of the decoded results in order from the beginning (e.g., the decoding order). In this case, the DCM order information indicates the output order of the decoded result of point M as "2", the output order of the decoded result of point J as "5", and the output order of the decoded result of point N as "6".
[0076] Also, for example, the DCM order information may indicate the output order of the decoded result by a difference value from the output order of the decoded result of the point to which the DCM that was applied was applied and output immediately before. In this case, the DCM order information indicates the output order of the decoded result of point M as "2" (a value obtained by subtracting the leading "0" from the output order "2" of the decoded result of point M). The DCM order information also indicates the output order of the decoded result of point J as "3" (a value obtained by subtracting the output order "2" of the decoded result of point M from the output order "5" of the decoded result of point J). Furthermore, the DCM order information indicates the output order of the decoded result of point N as "1" (a value obtained by subtracting the output order "5" of the decoded result of point J from the output order "6" of the decoded result of point N).
[0077] In this way, the amount of code for the DCM order information can be reduced compared to when the output order of the decoded results is indicated in order from the beginning (for example, the decoding order), which means that a decrease in coding efficiency can be suppressed.
[0078] Also, for example, the DCM order information may indicate the output order of the decoding result by a difference value from a predetermined reference order. For example, grid units may be provided that divide the result into a predetermined number of points, and the difference value may be initialized (e.g., set to "0") in each grid unit. In the example of FIG. 3, a dashed dotted line 23 indicates one grid unit. That is, in this example, a grid unit is provided every five points. When deriving a difference value from the point to which the previous DCM was applied, the difference value is reset if it exceeds such a grid unit.
[0079] In this case, the DCM order information indicates the output order of the decoded result of point M as "2" (a value obtained by subtracting the leading "0" from the output order "2" of the decoded result of point M). The DCM order information also indicates the output order of the decoded result of point J as "3" (a value obtained by subtracting the output order "2" of the decoded result of point M from the output order "5" of the decoded result of point J). The DCM order information also indicates the output order of the decoded result of point N as "1" (a value obtained by subtracting the grid unit "5" indicated by the dashed dotted line 23 from the output order "6" of the decoded result of point N). In FIG. 3, the difference value is represented as "1'" to indicate that it has been initialized.
[0080] In this way, the amount of code for the DCM order information can be reduced compared to when the output order of the decoded results is indicated by a difference value from the point where the previous DCM was applied, which makes it possible to suppress a decrease in coding efficiency.
[0081] The encoding side generates the above-mentioned DCM order information and transmits it to the decoding side. The decoding side acquires the transmitted DCM order information and controls the output order of the decoded results of the points to which DCM has been applied based on the DCM order information. That is, the decoded results of the points to which DCM has been applied are output in the correct output order. In other words, based on the DCM order information, the decoded results of the points to which DCM has been applied are inserted into the position of the correct output order in the string of decoded results of Octree-encoded data arranged in output order.
[0082] By doing this, even when DCM is applied, it is possible to eliminate the need for point rearrangement processing, and to suppress an increase in the load of decoding the encoded data of the point cloud.
[0083] <Middle layer control information> The encoded data of the above-mentioned tree-structured geometry data can be decoded in a scalable manner, i.e., the geometry data can be decoded at the resolution of a layer other than the lowest layer (a layer higher than the lowest layer).
[0084] However, the position (output order) of the points to which the above-mentioned DCM is applied may change depending on the layer. For example, as shown in Fig. 4, the decoded results of each node (point) in the layer surrounded by the solid-line square 31 are output in the order shown by the arrow 32.
[0085] In this case, the output order of point M, point J (node c), and point N to which DCM is applied differs from that in FIG.
[0086] Therefore, control information (DCM order information) may be generated on the encoding side even in intermediate hierarchies of such a tree structure, as in "Method 1-5" described in the sixth row from the top of the table shown in Fig. 2. For example, for a point where DCM is applied, control information (DCM order information) may be generated for all hierarchies (or some of the hierarchies) from the node to be processed where encoding with DCM is performed up to that point.
[0087] Such control information (DCM order information) may be signaled (transmitted from the encoding side to the decoding side). Furthermore, on the decoding side, the output order of the decoded results of the points to which DCM is applied may be controlled based on such control information (DCM order information).
[0088] In other words, the control information (DCM order information) may also include information regarding the output order of the decoding results in the intermediate layer of the tree structure.
[0089] In the case of this intermediate layer, the control information may also represent the output order of the decoded results, as in the case of the lowest layer described above. For example, the control information may indicate the output order of the decoded results of the points to which DCM is applied in order from the top (e.g., the decoding order). In the example of Fig. 4, the control information in this case indicates the output order of the decoded result of point M as "1", the output order of the decoded result of point J as "3", and the output order of the decoded result of point N as "4".
[0090] Also, for example, the control information may indicate the output order of the decoded result of a point to which DCM has been applied by a difference value from the output order of the decoded result of the point to which DCM has been applied that is output immediately before. In the example of Fig. 4, the control information in this case indicates the output order of the decoded result of point M as "1" (a value obtained by subtracting the leading "0" from the output order "1" of the decoded result of point M), the output order of the decoded result of point J as "2" (a value obtained by subtracting the output order "1" of the decoded result of point M from the output order "3" of the decoded result of point J), and the output order of the decoded result of point N as "1" (a value obtained by subtracting the output order "3" of the decoded result of point J from the output order "4" of the decoded result of point N).
[0091] In this way, the amount of code for the control information can be reduced compared to when the output order of the decoded results is indicated in order from the beginning (for example, the decoding order), which means that a decrease in coding efficiency can be suppressed.
[0092] Also, for example, the control information of a point to which DCM has been applied may indicate the output rank of the decoded result by a difference value from a predetermined reference rank. For example, grid units may be provided that divide the point into a predetermined number of points, and the difference value may be initialized (e.g., set to "0") in the grid unit. In the example of FIG. 4, a dashed dotted line 33 indicates one grid unit. That is, in this example, a grid unit is provided for every three points. When deriving a difference value from the previous point to which DCM has been applied, the difference value is reset if it exceeds such a grid unit.
[0093] In the example of Fig. 4, the control information in this case indicates the output order of the decoded result of point M as "1" (a value obtained by subtracting the leading "0" from the output order "1" of the decoded result of point M), the output order of the decoded result of point J as "2" (a value obtained by subtracting the output order "1" of the decoded result of point M from the output order "3" of the decoded result of point J), and the output order of the decoded result of point N as "1" (a value obtained by subtracting the grid unit "3" indicated by the dashed dotted line 23 from the output order "4" of the decoded result of point N). In Fig. 4, the difference values are represented as "1'" to indicate that they have been initialized.
[0094] In this way, the amount of code for the control information can be reduced compared to when the output order of the decoded results is expressed by a difference value from the point where the previous DCM was applied, which means that the reduction in coding efficiency can be suppressed.
[0095] As shown in A of Fig. 5, in the case of a conventional geometry data bit stream 40, coded data 41 of a point coded by applying DCM and coded data 42 of a point coded by Octree are configured as different data from each other and are not arranged in the processing order. In addition, there is no information indicating the processing order of each point. Furthermore, the Octree coded points are not arranged in the processing order. Therefore, it is necessary to sort the decoded results of the coded data of each point in the processing order of the attribute data (e.g., Morton order).
[0096] In contrast, by applying the above-described technology, as shown in B of Fig. 5, the bit stream 50 has coded data 51 of points coded using DCM, coded data 52 of points coded using Octree, and DCM order information 53. This DCM order information 53 indicates the output order of the decoded results of each point coded using DCM. Therefore, on the decoding side, based on this DCM order information 53, the decoded results of each point coded using DCM are inserted into the correct position of the Octree-coded points.
[0097] In this way, sorting of the decoded results of the geometry data becomes unnecessary, and an increase in the load of the decoding process of the encoded data of the point cloud can be suppressed.
[0098] <Control information transmission method> The transmission method of the control information as described above is arbitrary. That is, the control information may be transmitted by a predetermined transmission method, such as "Method 1-6" described in the seventh row from the top of the table shown in FIG. 2. For example, the control information may be included in the encoded data of the geometry data. Also, for example, the control information may be transmitted as data different from the encoded data of the geometry data. In that case, the control information may be associated with the encoded data of the geometry data using identification information or the like. For example, the control information may include identification information of the geometry data of the corresponding data unit (e.g., slice, etc.).
[0099] <2. First embodiment> <Encoding device> Next, a device to which the present technology described above in <1. Guarantee of the output order of the decoding results> is applied will be described. Fig. 6 is a block diagram showing an example of a configuration of an encoding device, which is one aspect of an information processing device to which the present technology is applied. The encoding device 100 shown in Fig. 6 is a device that encodes a point cloud (3D data). The encoding device 100 encodes the point cloud by applying the present technology described above in <1. Guarantee of the output order of the decoding results>.
[0100] Note that Fig. 6 shows the main processing units, data flows, etc., and is not limited to all that is shown in Fig. 6. In other words, in encoding device 100, there may be processing units that are not shown as blocks in Fig. 6, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 6.
[0101] As shown in FIG. 6, the encoding device 100 includes a position information encoding unit 101, a position information decoding unit 102, a point cloud generating unit 103, an attribute information encoding unit 104, and a bit stream generating unit 105.
[0102] The position information encoding unit 101 encodes geometry data (position information) of a point cloud (3D data) input to the encoding device 100. For example, the position information encoding unit 101 hierarchically organizes the geometry data to generate an octree, and encodes the octree. In addition, for example, the position information encoding unit 101 encodes geometry data of sparse points by applying DCM.
[0103] At that time, the position information encoding unit 101 applies the present technology described above in <1. Guarantee of output order of decoding results> to encode the geometry data. For example, the position information encoding unit 101 converts the geometry data into an octree and encodes it so that the decoding results of the geometry data are output in the processing order of the attribute data. The position information encoding unit 101 supplies the generated encoded data of the geometry data to the position information decoding unit 102 and the bit stream generating unit 105.
[0104] The position information decoding unit 102 obtains the encoded data of the geometry data supplied from the position information encoding unit 101, and decodes the encoded data. Any decoding method may be used as long as it is compatible with the encoding by the position information encoding unit 101. For example, processing such as filtering and inverse quantization for denoising may be performed. The position information decoding unit 102 supplies the generated geometry data (decoded result) to the point cloud generating unit 103.
[0105] The point cloud generating unit 103 acquires attribute data (attribute information) of the point cloud input to the encoding device 100 and geometry data (decoded result) supplied from the position information decoding unit 102. The point cloud generating unit 103 performs processing (recolor processing) to match the attribute data with the geometry data (decoded result). The point cloud generating unit 103 supplies the attribute data associated with the geometry data (decoded result) to the attribute information encoding unit 104.
[0106] In addition, the point cloud generation unit 103 applies the present technology described above in <1. Guarantee of the output order of the decoding results>, generates DCM order information indicating the output order of the decoding results of the points to which DCM is applied, and performs encoding processing. The point cloud generation unit 103 supplies the encoded data of the generated DCM order information to the bit stream generation unit 105.
[0107] The attribute information encoding unit 104 acquires the geometry data (decoded result) and attribute data supplied from the point cloud generating unit 103. The attribute information encoding unit 104 uses the geometry data (decoded result) to encode the attribute data and generate encoded data of the attribute data. The attribute information encoding unit 104 supplies the generated encoded data of the attribute data to the bit stream generating unit 105.
[0108] The bitstream generation unit 105 obtains coded data of the geometry data supplied from the position information encoding unit 101. The bitstream generation unit 105 also obtains coded data of the attribute data supplied from the attribute information encoding unit 104. The bitstream generation unit 105 also obtains coded data of the DCM order information supplied from the point cloud generation unit 103. The bitstream generation unit 105 generates a bitstream including these coded data. The bitstream generation unit 105 outputs the generated bitstream to the outside of the encoding device 100.
[0109] With this configuration, the encoding device 100 does not need to rearrange the points, and therefore can suppress an increase in the load of the decoding process of the encoded data of the point cloud. Therefore, the encoding device 100 can suppress an increase in implementation costs for reliably realizing real-time processing of the decoding of the encoded data of the point cloud.
[0110] Each of these processing units (position information encoding unit 101 to bit stream generating unit 105) of the encoding device 100 has an arbitrary configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0111] <Position information encoder> Fig. 7 is a block diagram showing a main configuration example of the position information encoding unit 101 (Fig. 6). Note that Fig. 7 shows main processing units, data flows, etc., and is not limited to all that are shown in Fig. 7. In other words, in the position information encoding unit 101, there may be processing units not shown as blocks in Fig. 7, and there may be processing and data flows not shown as arrows, etc. in Fig. 7.
[0112] As shown in FIG. 7, the position information encoding unit 101 has a bounding box setting unit 111, a voxel setting unit 112, a mode selection unit 113, an Octree encoding unit 114, and a DCM encoding unit 115.
[0113] The bounding box setting unit 111 performs processing related to setting of a bounding box. For example, the bounding box setting unit 111 acquires geometry data of the point cloud data input to the encoding device 100. The bounding box setting unit 111 sets a bounding box for the geometry data. The bounding box is information for normalizing the geometry data to be encoded. Voxelization is performed based on this bounding box. The bounding box setting unit 111 supplies information related to the bounding box to the voxel setting unit 112 together with the geometry data.
[0114] The voxel setting unit 112 performs processing related to voxel setting. For example, the voxel setting unit 112 acquires information related to the geometry data and the bounding box supplied from the bounding box setting unit 111. Based on the information, the voxel setting unit 112 divides the bounding box set for the geometry data to set voxels. That is, the voxel setting unit 112 voxels the geometry data (quantizes the position of each point). The voxel setting unit 112 supplies voxel data, which is the geometry data voxelized in this way, to the mode selection unit 113.
[0115] The mode selection unit 113 performs processing related to the selection of an encoding method (mode). For example, the mode selection unit 113 acquires voxel data supplied from the voxel setting unit 112. The mode selection unit 113 also selects an encoding method (mode) for each voxel (node in an Octree). That is, the mode selection unit 113 selects whether to perform Octree encoding or DCM-applied encoding (also referred to as DCM encoding) on the voxel to be processed.
[0116] For example, the mode selection unit 113 determines whether a voxel to be processed is sparse. If it is determined that the voxel is not sparse based on a predetermined condition, the mode selection unit 113 selects Octree encoding as the encoding method, and supplies the voxel data to be processed to the Octree encoding unit 114. If it is determined that the voxel is sparse based on a predetermined condition, the mode selection unit 113 selects DCM encoding as the encoding method, and supplies the voxel data to be processed to the DCM encoding unit 115.
[0117] The Octree encoding unit 114 performs processing related to encoding using an Octree. For example, the Octree encoding unit 114 obtains voxel data to be processed that is supplied from the mode selection unit 113. The Octree encoding unit 114 uses the voxel data to generate Octree data (Child Mask) of the node to be processed. The Octree encoding unit 114 encodes the Octree data of the node to be processed using a predetermined method to generate encoded data.
[0118] At this time, the Octree encoding unit 114 performs encoding by applying the present technology (for example, "Method 1" in FIG. 2 (which may include Methods 1-1 to 1-3)) described above in <1. Guarantee of Output Order of Decoded Results>. That is, the Octree encoding unit 114 encodes the Octree data of the processing target node such that the decoded results of the geometry data are output in the processing order of the attribute data. For example, the Octree encoding unit 114 encodes the Octree data of the processing target node in an order such that the decoded results of the geometry data are output in Morton order. For example, the Octree encoding unit 114 encodes the Octree data of the processing target node in an order such that the decoded results of the geometry data are output in Morton order at each node of the Octree in the processing order of the attribute data of the point cloud.
[0119] The Octree encoding unit 114 supplies the encoded data generated by performing the encoding in this manner (encoded data of the voxel data of the node to be processed) to the position information decoding unit 102 and the bit stream generating unit 105 (both in FIG. 6).
[0120] The DCM encoding unit 115 performs processing related to encoding using DCM. For example, the DCM encoding unit 115 obtains voxel data to be processed that is supplied from the mode selection unit 113. The DCM encoding unit 115 applies DCM to encode the voxel data to generate encoded data. For example, the DCM encoding unit 115 uses the voxel data to encode the relative distance from the node to be processed to the leaf to generate encoded data. The DCM encoding unit 115 supplies the generated encoded data to the position information decoding unit 102 and the bit stream generation unit 105 (both in FIG. 6).
[0121] With this configuration, the position information encoding unit 101 can perform Octree encoding so as to ensure that the output order of the decoded results of the geometry data is a predetermined order. Therefore, the encoding device 100 does not need to rearrange the points, and can suppress an increase in the load of the decoding process of the encoded data of the point cloud.
[0122] Each of these processing units (bounding box setting unit 111 to DCM encoding unit 115) of the position information encoding unit 101 has an arbitrary configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0123] <Point cloud generation section> Fig. 8 is a block diagram showing a main configuration example of the point cloud generating unit 103 (Fig. 6). Note that Fig. 8 shows main processing units, data flows, etc., and is not limited to all that are shown in Fig. 8. In other words, in the point cloud generating unit 103, there may be processing units that are not shown as blocks in Fig. 8, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 8.
[0124] As shown in FIG. 8, the point cloud generating unit 103 has a Morton code conversion unit 121, a rearrangement unit 122, a DCM order information generating unit 123, a DCM order information encoding unit 124, and a recolor processing unit 125.
[0125] The Morton code conversion unit 121 converts the geometry data (decoded result) supplied from the position information decoding unit 102 (FIG. 6) into a Morton code. In other words, the Morton code conversion unit 121 maps points in a three-dimensional space into one dimension using the Morton code. The Morton code conversion unit 121 supplies the geometry data of each point with the Morton code to the rearrangement unit 122.
[0126] The rearrangement unit 122 obtains the geometry data supplied from the Morton code conversion unit 121 and sorts the data based on the value of the Morton code. That is, the rearrangement unit 122 rearranges the geometry data of each point into the Morton order. The rearrangement unit 122 supplies the geometry data of each point rearranged into the Morton order to the DCM order information generation unit 123 and the recolor processing unit 125.
[0127] The DCM order information generation unit 123 identifies points to which DCM is applied during encoding among the geometry data of each point rearranged in Morton order, and generates DCM order information, which is control information that indicates the output order of the decoded results of those points.
[0128] For example, the DCM order information may include information indicating the output order of the decoded results of the point to which the DCM has been applied, in order from the beginning.The DCM order information may also include information indicating the output order of the decoded results of the point to which the DCM has been applied, as a difference value from the output order of the decoded results of the point to which the DCM has been applied that is output immediately before.Furthermore, the DCM order information may include information indicating the output order of the decoded results of the point to which the DCM has been applied, as a difference value from a predetermined reference order.
[0129] The DCM order information generating unit 123 may generate DCM order information indicating the output order of the points to which DCM is applied in the intermediate layer of the Octree as well. That is, the DCM order information may include information regarding the output order of the decoded results in the intermediate layer of the Octree. The DCM order information generating unit 123 supplies the generated DCM order information to the DCM order information encoding unit 124.
[0130] The DCM order information encoder 124 performs processing related to encoding of the DCM order information. For example, the DCM order information encoder 124 acquires DCM order information supplied from the DCM order information generator 123. The DCM order information encoder 124 encodes the acquired DCM order information to generate encoded data. Any encoding method may be used. The DCM order information encoder 124 supplies the encoded data of the generated DCM order information to the bit stream generator 105 (FIG. 6).
[0131] The recolor processing unit 125 performs processing related to a process of matching attribute data with geometry data (recolor processing). For example, the recolor processing unit 125 obtains attribute data of a point cloud input to the encoding device 100. The recolor processing unit 125 also obtains geometry data of each point rearranged in Morton order supplied from the rearrangement unit 122.
[0132] The recolor processing unit 125 performs processing (recolor processing) to match the acquired attribute data with the acquired geometry data, and generates point cloud data. The recolor processing unit 125 supplies the generated point cloud data to the attribute information encoding unit 104 (FIG. 6).
[0133] With this configuration, the point cloud generating unit 103 can generate DCM order information and transmit it to the decoding side. This DCM order information allows the decoded results of the points encoded by applying DCM to be output in an appropriate output order. Therefore, since it is not necessary to rearrange the points during decoding, the encoding device 100 can suppress an increase in implementation costs for reliably realizing real-time processing of decoding the encoded data of the point cloud.
[0134] Each of these processing units (Morton code conversion unit 121 to recolor processing unit 125) of the point cloud generating unit 103 has an arbitrary configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0135] <Encoding process flow> Next, a description will be given of the processing executed by the encoding device 100. The encoding device 100 encodes point cloud data by executing an encoding process. An example of the flow of this encoding process will be described with reference to the flowchart in FIG.
[0136] When the encoding process is started, in step S101, the position information encoding unit 101 of the encoding device 100 encodes the geometry data (position information) of the input point cloud to generate encoded data of the geometry data. At that time, the position information encoding unit 101 performs processing by applying the present technology described above in <1. Guarantee of output order of decoded results>. Details of the position information encoding process will be described later.
[0137] In step S102, the position information decoding unit 102 decodes the encoded data of the geometry data generated in step S101 to generate position information.
[0138] In step S103, the point cloud generating unit 103 performs recolor processing using the attribute data (attribute information) of the input point cloud and the geometry data (decoding result) generated in step S102, and generates point cloud data by associating the attribute data with the geometry data. In addition, at this time, the point cloud generating unit 103 applies the present technology described above in <1. Guarantee of output order of decoding results> to generate DCM order information indicating the output order of the decoding results of points to which DCM is applied, and performs encoding processing. Details of the point cloud generation processing will be described later.
[0139] In step S104, the attribute information encoding unit 104 executes an attribute information encoding process to encode the attribute data that has been recolored in step S103, and generates encoded data of the attribute data.
[0140] In step S105, the bitstream generation unit 105 generates and outputs a bitstream including the encoded data of the geometry data generated in step S101, the encoded data of the DCM order information generated in step S103, and the encoded data of the attribute data generated in step S104.
[0141] When the process of step S105 ends, the encoding process ends.
[0142] By performing the processes at each step in this manner, the encoding device 100 can eliminate the need for point rearrangement processing during decoding, thereby suppressing an increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, the encoding device 100 can suppress an increase in implementation costs for reliably achieving real-time processing of the decoding of the encoded data of the point cloud.
[0143] <Location information encoding process flow> Next, an example of the flow of the position information encoding process executed in step S101 in FIG. 9 will be described with reference to the flowchart in FIG.
[0144] When the position information encoding process starts, in step S121, the bounding box setting unit 111 sets a bounding box for the geometry data to be processed.
[0145] In step S122, the voxel setting unit 112 sets voxels based on the bounding box set in step S121, and quantizes the geometry data of each point.
[0146] In step S123, the mode selection unit 113 selects voxel data to be processed from the voxels set in step S122 in accordance with the Morton order.
[0147] In step S124, the mode selection unit 113 determines whether or not to apply DCM to the voxel data to be processed. If it is determined to be sparse based on a predetermined condition, the process proceeds to step S125.
[0148] In step S125, the DCM encoding unit 115 DCM-encodes the voxel data to be processed. When the process of step S125 ends, the process proceeds to step S127.
[0149] If it is determined in step S124 that the DCM is not to be applied to the voxel data to be processed, the process proceeds to step S126.
[0150] In step S126, the Octree encoding unit 114 applies the present technology described above in <1. Guarantee of the output order of the decoding results> and performs Octree encoding on the voxel data to be processed so that the output order of the decoding results is the Morton order. When the process of step S126 ends, the process proceeds to step S127.
[0151] In step S127, the mode selection unit 113 determines whether all voxel data has been processed. If unprocessed voxel data exists, the process returns to step S123, and the subsequent processes are repeated. That is, the processes of steps S123 to S127 are executed for each voxel data.
[0152] Then, if it is determined in step S127 that all the voxel data has been processed, the position information encoding process ends and the process returns to FIG.
[0153] By performing the processes at each step in this manner, the encoding device 100 can eliminate the need for point rearrangement processing during decoding, thereby suppressing an increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, the encoding device 100 can suppress an increase in implementation costs for reliably achieving real-time processing of the decoding of the encoded data of the point cloud.
[0154] <Point cloud generation process flow> Next, an example of the flow of the point cloud generation process executed in step S103 in FIG. 9 will be described with reference to the flowchart in FIG.
[0155] When the point cloud generation process is started, the Morton code conversion unit 121 (FIG. 8) of the point cloud generation unit 103 converts the geometry data into Morton code in step S141.
[0156] In step S142, the rearrangement unit 122 rearranges the geometry data into Morton order based on the Morton code added in step S121.
[0157] In step S143, the DCM order information generating unit 123 identifies points to which the DCM is applied during encoding among the geometry data for each point rearranged in the Morton order, and generates DCM order information indicating the output order of the decoded results of the identified points. At this time, the DCM order information generating unit 123 generates the DCM order information by applying the present technology described above in <1. Guarantee of the output order of the decoded results>.
[0158] In step S144, the DCM order information encoding unit 124 encodes the DCM order information generated in step S143 to generate encoded data.
[0159] In step S145, the recolor processing unit 125 performs processing (recolor processing) to match the acquired attribute data with the acquired geometry data, and generates point cloud data.
[0160] When the process of step S145 ends, the point cloud generation process ends, and the process returns to FIG.
[0161] By performing the processes at each step in this manner, the encoding device 100 can eliminate the need for point rearrangement processing during decoding, thereby suppressing an increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, the encoding device 100 can suppress an increase in implementation costs for reliably achieving real-time processing of the decoding of the encoded data of the point cloud.
[0162] <3. Second embodiment> <Decryption device> Next, another example of a device to which the present technology described above in <1. Guarantee of the output order of the decoding results> is applied will be described. Fig. 12 is a block diagram showing an example of a configuration of a decoding device, which is one aspect of an information processing device to which the present technology is applied. The decoding device 200 shown in Fig. 12 is a device that decodes encoded data of a point cloud (3D data). The decoding device 200 decodes the encoded data of the point cloud by applying the present technology described above in <1. Guarantee of the output order of the decoding results>.
[0163] Note that Fig. 12 shows the main processing units, data flows, and the like, and is not limited to all that is shown in Fig. 12. In other words, in the decoding device 200, there may be processing units that are not shown as blocks in Fig. 12, and there may be processing and data flows that are not shown as arrows or the like in Fig. 12.
[0164] As shown in FIG. 12, the decoding device 200 includes an encoded data extraction unit 201, a position information decoding unit 202, an attribute information decoding unit 203, and a point cloud generation unit 204.
[0165] The coded data extraction unit 201 acquires and holds a bit stream input to the decoding device 200. The coded data extraction unit 201 extracts coded data of geometry data (position information) and attribute data (attribute information) from the bit stream. In this case, the coded data extraction unit 201 can extract coded data of all layers from the bit stream. In addition, for example, the coded data extraction unit 201 can also extract coded data from the top layer to a layer specified by a user, an application, or the like (that is, coded data of a part of layers) from the bit stream.
[0166] The encoded data extraction unit 201 supplies the extracted encoded data of the geometry data to the position information decoding unit 202. The encoded data extraction unit 201 supplies the extracted encoded data of the attribute data to the attribute information decoding unit 203.
[0167] The position information decoding unit 202 acquires the coded data of the geometry data supplied from the coded data extraction unit 201. The position information decoding unit 202 decodes the coded data of the geometry data to generate geometry data (decoded result). At that time, the position information decoding unit 202 decodes the geometry data by applying the present technology described above in <1. Guarantee of output order of decoded result>. In other words, the position information decoding unit 202 decodes the coded data of the geometry data and outputs the decoded result in the processing order of the attribute data of the point cloud.
[0168] The position information decoding unit 202 supplies the generated geometry data (decoded result) to the attribute information decoding unit 203 and the point cloud generating unit 204.
[0169] The attribute information decoding unit 203 acquires the coded data of the attribute data supplied from the coded data extraction unit 201. The attribute information decoding unit 203 acquires the geometry data (decoded result) supplied from the position information decoding unit 202. The attribute information decoding unit 203 uses the geometry data to decode the coded data of the attribute data and generate attribute data (decoded result). The attribute information decoding unit 203 supplies the generated attribute data (decoded result) to the point cloud generation unit 204.
[0170] The point cloud generating unit 204 acquires geometry data (decoding result) supplied from the position information decoding unit 202. The point cloud generating unit 204 acquires attribute data (decoding result) supplied from the attribute information decoding unit 203. The point cloud generating unit 204 generates a point cloud (decoding result) using the geometry data (decoding result) and the attribute data (decoding result). The point cloud generating unit 204 outputs the generated point cloud (decoding result) data to the outside of the decoding device 200.
[0171] With this configuration, the geometry data of the decoded result is output in the same order as the attribute data is processed, so that the point rearrangement process is not required. Therefore, the decoding device 200 can suppress an increase in the load of the decoding process of the encoded data of the point cloud. Therefore, it is possible to suppress an increase in the implementation cost for reliably realizing the real-time process of decoding the encoded data of the point cloud.
[0172] In addition, these processing units (the encoded data extraction unit 201 to the point cloud generation unit 204) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0173] <Location Information Decoding Unit> Fig. 13 is a block diagram showing a main configuration example of the position information decoding unit 202 (Fig. 12). Note that Fig. 13 shows main processing units, data flows, etc., and is not limited to all of the processing units shown in Fig. 13. In other words, in the position information decoding unit 202, there may be processing units not shown as blocks in Fig. 13, and there may be processing and data flows not shown as arrows in Fig. 13.
[0174] As shown in FIG. 13, the position information decoding unit 202 includes a decoding unit 211 and a DCM insertion unit 212 .
[0175] The decoding unit 211 performs processing related to decoding of geometry data. For example, the position information decoding unit 202 acquires the encoded data of the geometry data supplied from the encoded data extraction unit 201. The decoding unit 211 decodes the acquired encoded data to generate (restore) the geometry data.
[0176] For example, the decoding unit 211 decodes the coded data of DCM order information included in the coded data of geometry data. Also, for example, the decoding unit 211 decodes the coded data of Octree data into Morton order. Furthermore, for example, the decoding unit 211 decodes coded data that has been DCM coded. The decoding unit 211 supplies the geometry data generated by decoding (which may include DCM order information, Octree data, geometry data of points to which DCM has been applied (also referred to as DCM data), etc.) to the DCM insertion unit 212.
[0177] The DCM insertion unit 212 performs processing related to control of the output order of geometry data of points to which a DCM is applied. For example, the DCM insertion unit 212 obtains geometry data (which may include DCM order information, Octree data, DCM data, etc.) supplied from the decoding unit 211.
[0178] The DCM insertion unit 212 inserts the acquired DCM data into the column of the Octree data rearranged in Morton order at the position of the output order indicated by the DCM order information. In other words, the DCM insertion unit 212 controls the DCM data so that it is output in the output order indicated by the DCM order information corresponding to that DCM data.
[0179] The DCM insertion unit 212 supplies the octree data into which the DCM data has been inserted (geometry data rearranged in Morton order) to the point cloud generation unit 204 (FIG. 12).
[0180] With this configuration, the geometry data of the decoded result is output in the same order as the attribute data is processed, so that the point rearrangement process is not required. Therefore, the decoding device 200 can suppress an increase in the load of the decoding process of the encoded data of the point cloud. Therefore, it is possible to suppress an increase in the implementation cost for reliably realizing the real-time process of decoding the encoded data of the point cloud.
[0181] These processing units (the decoding unit 211 and the DCM insertion unit 212) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and by executing a program.
[0182] <Decryption process flow> Next, a description will be given of the processing executed by the decoding device 200. The decoding device 200 decodes the encoded data of the point cloud by executing a decoding process. An example of the flow of the decoding process will be described with reference to the flowchart in FIG.
[0183] When the decoding process is started, in step S201, the encoded data extraction unit 201 of the decoding device 200 acquires and stores a bitstream, and extracts from it encoded data of the geometry data (position information) and attribute data (attribute information) of the layer to be decoded.
[0184] In step S202, the position information decoding unit 202 decodes the encoded data of the geometry data extracted in step S201 to generate geometry data (decoded result). At this time, the position information decoding unit 202 performs processing by applying the present technology described above in <1. Guarantee of output order of decoded result>. Details of the position information decoding process will be described later.
[0185] In step S203, the attribute information decoding unit 203 decodes the coded data of the attribute data extracted in step S201, and generates attribute data (decoded result).
[0186] In step S204, the point cloud generating unit 204 generates and outputs point cloud data (decoding result) using the geometry data (decoding result) generated in step S202 and the attribute data (decoding result) generated in step S203.
[0187] When the process of step S204 ends, the decoding process ends.
[0188] By performing the processing of each step in this manner, the geometry data of the decoded result is output in the same order as the processing order of the attribute data, so that the rearrangement processing of the points is not necessary. Therefore, the decoding device 200 can suppress the increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, it is possible to suppress the increase in the implementation cost for reliably realizing the real-time processing of the decoding of the encoded data of the point cloud.
[0189] <Location information decoding process flow> Next, an example of the flow of the position information decoding process executed in step S202 in FIG. 14 will be described with reference to the flowchart in FIG.
[0190] When the position information decoding process starts, the decoding unit 211 of the position information decoding unit 202 decodes the DCM order information in step S221.
[0191] In step S222, the decoding unit 211 decodes the encoded data of the Octree data into Morton order.
[0192] In step S223, the decoding unit 211 decodes the encoded data of the DCM data.
[0193] In step S224, the DCM insertion unit 212 inserts the DCM data decoded in step S223 into the Morton-ordered Octree data decoded in step S222 at a position corresponding to the output order indicated by the DCM order information decoded in step S221. That is, the DCM insertion unit 212 controls the output order of the geometry data (decoded results) so that the DCM data is output in the output order indicated by the DCM order information. At this time, the DCM insertion unit 212 performs processing by applying the present technology described above in <1. Guarantee of output order of decoded results>.
[0194] When the process of step S224 ends, the position information decoding process ends, and the process returns to FIG.
[0195] By performing the processing of each step in this manner, the geometry data of the decoded result is output in the same order as the processing order of the attribute data, so that the rearrangement processing of the points is not necessary. Therefore, the decoding device 200 can suppress the increase in the load of the decoding processing of the encoded data of the point cloud. Therefore, it is possible to suppress the increase in the implementation cost for reliably realizing the real-time processing of the decoding of the encoded data of the point cloud.
[0196] <4. Notes> <Other output order> In the above, it has been described that the decoded results of the geometry data are output in the processing order of the attribute data, but the output order of the decoded results of the geometry data may be any known order and is not limited to the above example. If the output order is known, it is possible to perform processing suitable for that output order in the subsequent stage, and rearrangement of the points may be unnecessary.
[0197] For example, if the attribute data is not decoded, the output order of the decoded results of the geometry data may be an order other than the processing order of the attribute data.
[0198] 16, when decoding geometry data of a partial region 302 that is a 3D region that is part of a bounding box 301 that contains an object, if the decoded result of the geometry data is output in Morton order, it is possible to specify the geometry data included in the partial region 302 as a group of data. In other words, it is possible to easily decode only the geometry data within the partial region 302 by simply specifying the first point (Start number) and the last point (End number) as the decoding target.
[0199] In this way, by applying the present technology described above in <1. Guarantee of the output order of the decoded results> to perform encoding and decoding, and outputting the decoded results of the geometry data in a predetermined order, the rearrangement process of the points may become unnecessary. Therefore, in such a case, it is possible to suppress an increase in the load of the decoding process of the encoded data of the point cloud. Therefore, it is possible to suppress an increase in the implementation cost for reliably realizing the real-time processing of the decoding of the encoded data of the point cloud.
[0200] <Control information> Although the signaling of control information has been described above, control information other than the above-mentioned example may be signaled. For example, control information (e.g., enabled_flag) that controls whether or not to permit (or prohibit) application of the present technology described above may be transmitted. In addition, for example, control information that specifies a range (e.g., upper or lower limit of block size, or both, slice, picture, sequence, component, view, layer, etc.) in which application of the present technology described above is permitted (or prohibited) may be transmitted.
[0201] <Computer> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, etc., capable of executing various functions by installing various programs.
[0202] FIG. 17 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.
[0203] In a computer 900 shown in FIG. 17, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.
[0204] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.
[0205] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0206] In the computer configured as above, the series of processes described above are performed by the CPU 901, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data and the like necessary for the CPU 901 to execute various processes as appropriate.
[0207] The program executed by the computer can be applied by recording it on a removable medium 921 such as a package medium, for example. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by mounting the removable medium 921 on the drive 915.
[0208] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.
[0209] In addition, this program can also be installed in advance in the ROM 902 or the storage unit 913.
[0210] <Target of application of this technology> Although the above describes the application of this technology to the encoding and decoding of point cloud data, this technology is not limited to these examples and can be applied to the encoding and decoding of 3D data of any standard. In other words, as long as it does not conflict with the above-mentioned technology, various processes such as encoding and decoding methods and specifications of various data such as 3D data and metadata are arbitrary. In addition, as long as it does not conflict with the present technology, some of the above-mentioned processes and specifications may be omitted.
[0211] Furthermore, although the encoding device 100 and the decoding device 200 have been described above as application examples of the present technology, the present technology can be applied to any configuration.
[0212] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) in satellite broadcasting, wired broadcasting such as cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media.
[0213] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).
[0214] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and cooperate with each other to perform processing via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (video images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.
[0215] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.
[0216] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications are also arbitrary.
[0217] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Thus, the value that this "flag" can take may be, for example, a binary value of 1 / 0, or may be three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, it is assumed that identification information (including flags) is not only included in the bit stream, but also includes difference information of the identification information with respect to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information with respect to the reference information.
[0218] Also, various information (metadata, etc.) related to the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, data associated with each other may be combined into one piece of data, or each piece of data may be made into separate data. For example, information associated with coded data (image) may be transmitted on a transmission path different from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium different from that of the coded data (image) (or on a different recording area of the same recording medium). Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in any unit, such as multiple frames, one frame, or a part of a frame.
[0219] In this specification, the terms "combine," "multiplex," "append," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.
[0220] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.
[0221] For example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).
[0222] Also, for example, the above-mentioned program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.
[0223] Also, for example, each step of one flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.
[0224] Also, for example, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel, or individually at a required timing such as when a call is made. In other words, the processing of each step may be executed in an order different from the order described above, so long as no contradiction occurs. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.
[0225] Also, for example, a plurality of techniques related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any of a plurality of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in other embodiments. Also, a part or all of any of the above-mentioned present technologies can be implemented in combination with other technologies not described above.
[0226] The present technology can also be configured as follows. (1) an encoding unit that encodes position information of a point cloud that represents a three-dimensional object as a set of points in an order such that a decoded result of the encoded data of the position information is output in the processing order of attribute information of the point cloud, and generates the encoded data; An information processing device comprising: (2) The encoding unit encodes the position information in an order such that the decoded result is output in Morton order, and generates the encoded data. An information processing device as described in (1). (3) The encoding unit encodes the tree-structured position information in an order such that the decoded results are output in a processing order of the attribute information of the point cloud at each node of the tree structure, and generates the encoded data. An information processing device according to (1) or (2). (4) The tree structure is an octree. (3) An information processing device. (5) a DCM encoding unit that encodes the position information of the sparse points of the octree by applying a direct coding mode (DCM) to generate encoded data of the position information; a generation unit that generates control information regarding an output order of a decoding result of the encoded data generated by the DCM encoding unit; The information processing device according to (4) further comprises: (6) The control information includes information indicating an output order of the decoded results from the beginning. (5) An information processing device according to the present invention. (7) The control information includes information indicating an output order of the decoding result by a difference value from an output order of the decoding result of the coded data of the position information coded by applying the DCM output immediately before. (5) An information processing device according to the present invention. (8) The control information includes information indicating an output rank of the decoded result by a difference value from a predetermined reference rank. (5) An information processing device according to the present invention. (9) The control information also includes information regarding an output order of the decoded results in an intermediate layer of the tree structure. An information processing device according to any one of (5) to (8). (10) Encoding position information of a point cloud that represents a three-dimensional object as a set of points in an order such that a decoded result of the encoded data of the position information is output in the processing order of the attribute information of the point cloud, and generating the encoded data. Information processing methods.
[0227] (11) A decoding unit that decodes encoded data of position information of a point cloud that represents a three-dimensional object as a set of points, and outputs the decoded result in the processing order of the attribute information of the point cloud. An information processing device comprising: (12) The decoding unit outputs the decoding result in Morton order. An information processing device according to (11). (13) The decoding unit decodes the encoded data of the position information in a tree structure, and outputs the decoding result in each node of the tree structure in a processing order of the attribute information of the point cloud. The information processing device according to (11) or (12). (14) The tree structure is an octree. An information processing device according to (13). (15) The device further includes an output control unit that outputs a result of decoding the coded data of the position information coded by applying DCM (Direct Coding Mode) in an output order indicated by control information regarding an output order of the result of decoding the coded data. The information processing device according to (14) further comprising: (16) The control information includes information indicating an output order of the decoded results from the beginning. An information processing device according to (15). (17) The control information includes information indicating an output order of the decoding result by a difference value from an output order of a decoding result of encoded data of the position information that is encoded by applying the DCM and output immediately before. An information processing device according to (15). (18) The control information includes information indicating an output rank of the decoded result by a difference value from a predetermined reference rank. An information processing device according to (15). (19) The control information also includes information regarding an output order of the decoded results in an intermediate layer of the tree structure. An information processing device according to any one of (15) to (18). (20) Decode encoded data of position information of a point cloud that represents a three-dimensional object as a collection of points, and output the decoded results in the processing order of the attribute information of the point cloud. Information processing methods. [Explanation of symbols]
[0228] 100 encoding device, 101 position information encoding unit, 102 position information decoding unit, 103 point cloud generation unit, 104 attribute information encoding unit, 105 bitstream generation unit, 111 bounding box setting unit, 112 voxel setting unit, 113 mode selection unit, 114 Octree encoding unit, 115 DCM encoding unit, 121 Morton code conversion unit, 122 rearrangement unit, 123 DCM order information generation unit, 124 DCM order information encoding unit, 125 recolor processing unit, 200 decoding device, 201 encoded data extraction unit, 202 position information decoding unit, 203 attribute information decoding unit, 204 point cloud generation unit, 211 decoding unit, 212 DCM insertion unit< / dcm> < / lifting> < / octree>
Claims
1. an Octree encoding unit that converts position information of a point cloud that expresses a three-dimensional object as a set of points into an Octree, encodes the position information in an order in which the decoded results are output in the processing order of attribute information of the point cloud, and generates first encoded data; a DCM encoding unit that encodes position information of the sparsely populated portion of the point cloud using a direct coding mode (DCM) to generate second encoded data; a generating unit that generates control information regarding an output order of a decoding result of the generated second encoded data; An information processing device comprising:
2. The Octree encoding unit encodes the position information in an order such that the decoded result is output in Morton order, and generates the first encoded data. The information processing device according to claim 1 .
3. The control information includes information indicating an output order of the decoded results from the beginning. The information processing device according to claim 1 .
4. The control information includes information indicating an output rank of the decoding result by a difference value from an output rank of the decoding result of the second encoded data output immediately before. The information processing device according to claim 1 .
5. The control information includes information indicating an output rank of the decoded result by a difference value from a predetermined reference rank. The information processing device according to claim 1 .
6. The control information also includes information regarding the output order of the decoding results in an intermediate layer of the octree. The information processing device according to claim 1 .
7. Converting position information of a point cloud, which represents an object having a three-dimensional shape as a set of points, into an octree, and encoding the position information in an order such that a decoded result is output in the processing order of attribute information of the point cloud, thereby generating first encoded data; encoding position information of the sparsely populated portion of the point cloud using a direct coding mode (DCM) to generate second encoded data; generating control information relating to an output order of the decoded results of the generated second encoded data; An information processing method comprising:
8. A decoding unit which decodes first encoded data in which positional information of a point cloud which represents a three-dimensional object as a collection of points is octree-coded and encoded, outputs a result of the decoding of the first encoded data in the processing order of attribute information of the point cloud, and further decodes second encoded data in which positional information of a part of the point cloud where the points are sparsely coded using DCM (Direct Coding Mode); an output control unit that outputs a result of decoding the second encoded data in an output order indicated by control information regarding an output order of the result of decoding the second encoded data; An information processing device comprising:
9. The decoding unit outputs the decoded result in Morton order. The information processing device according to claim 8.
10. The control information includes information indicating an output order of the decoded results from the beginning. The information processing device according to claim 8.
11. The control information includes information indicating an output rank of the decoding result by a difference value from an output rank of the decoding result of the second encoded data output immediately before. The information processing device according to claim 8.
12. The control information includes information indicating an output rank of the decoded result by a difference value from a predetermined reference rank. The information processing device according to claim 8.
13. The control information also includes information regarding an output order of the decoding results in an intermediate layer of the octree. The information processing device according to claim 8.
14. Decoding first encoded data in which positional information of a point cloud representing a three-dimensional object as a collection of points is octree-coded and encoded, outputting the decoded result of the first encoded data in the processing order of attribute information of the point cloud, and further decoding second encoded data in which positional information of a portion of the point cloud in which the points are sparsely coded using DCM (Direct Coding Mode); outputting a result of the decoding of the second encoded data in an output order indicated by control information regarding an output order of the result of the decoding of the second encoded data; An information processing method comprising:
Citation Information
Patent Citations
Encoder and control method therefor
JP2017126890A
Point Cloud Compression
US20190080483A1
Three-dimensional data generation method, three-dimensional data transmission method, three-dimensional data generation device, and three-dimensional data transmission device
WO2018016168A1
Information processing device and method
WO2019069711A1
Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2019082837A1
