Decryption processing unit and method
By layering and sub-hierarchizing attribute information in 3D point cloud data, the method addresses biased prediction accuracy issues, enhancing encoding efficiency and maintaining accurate prediction in 3D data encoding and decoding.
Patent Information
- Application Number
- JP2024140694
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2019-07-12
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-06-26
AI Technical Summary
Existing methods for encoding 3D point cloud data suffer from reduced encoding efficiency due to biased prediction accuracy caused by decoding attribute data in a specific direction within the three-dimensional space, limiting the reference to attribute data located before the target data.
The method involves layering and sub-hierarchizing attribute information of a point cloud to generate reference relationships between layers and sub-layers, reducing bias in prediction direction and enhancing encoding efficiency.
This approach suppresses reduction in encoding efficiency by minimizing bias in the prediction direction and maintaining accurate prediction accuracy, allowing for efficient encoding and decoding of 3D point cloud data.
Smart Images

Figure 2024164140000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method capable of suppressing a decrease in encoding efficiency. [Background technology]
[0002] Conventionally, a method for encoding 3D data representing a three-dimensional structure such as a point cloud has been considered (see, for example, Non-Patent Document 1). Point cloud data is composed of geometry data (also called position information) and attribute data (also called attribute information) of each point. Therefore, encoding of a point cloud is performed for each of its geometry data and attribute data. Various methods have been proposed as a method for encoding attribute data. For example, a technique called Lifting has been proposed (see, for example, Non-Patent Document 2). In addition, a method for enabling attribute data to be decoded in a scalable manner has also been proposed (see, for example, Non-Patent Document 3). In addition, a method other than Lifting that allows decoded attribute data to be referenced within the LoD has also been considered (see, for example, Non-Patent Document 4). [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] R. Mekuria, Student Member IEEE, K. Blom, P. Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video",tcsvt_paper_submitted_february.pdf [Non-Patent Document 2] Khaled Mammou, Alexis Tourapis, Jungsun Kim, Fabrice Robinet, Valery Valentin, Yeping Su, "Lifting Scheme for Lossy Attribute Encoding in TMC1", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42640, April 2018, San Diego, US [Non-Patent Document 3] Ohji Nakagami, Satoru Kuma, "[G-PCC] Spatial scalability support for G-PCC", ISO / IEC JTC1 / SC29 / WG11 MPEG2019 / m47352, March 2019, Geneva, CH [Non-Patent Document 4] Toshiyasu Sugio, "[G-PCC] Reference structure modification on attribute predicting transform in TMC13", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m46107, January 2019, Marrakech, MA Summary of the Invention [Problem to be solved by the invention]
[0004] However, in the case of the technique described in Non-Patent Document 4, attribute data is arranged in Morton Order within the LoD and encoded, so that the decoded attribute data that can be referred to is limited to that located before the attribute data to be decoded in the Morton Order. In other words, there is a risk that the decoded attribute data is limited to attribute data of a point located in a biased direction from the point of the attribute data to be decoded in three-dimensional space. Therefore, there is a risk that the prediction accuracy of the attribute data to be processed, which is performed by referring to the decoded attribute data, is reduced, and the encoding efficiency is reduced.
[0005] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress a decrease in coding efficiency. [Means for solving the problem]
[0006] An information processing device according to one aspect of the present technology is an information processing device including a hierarchical structure generating unit that hierarchically organizes attribute information of a point cloud that represents an object having a three-dimensional shape as a collection of points, and generates a reference relationship of the attribute information between hierarchies, and a sub-hierarchical structure generating unit that sub-organizes the attribute information of the hierarchy of the attribute information generated by the hierarchical structure generating unit, and generates a reference relationship of the attribute information between the sub-hierarchies.
[0007] An information processing method according to one aspect of the present technology is an information processing method that hierarchically organizes attribute information of a point cloud that represents a three-dimensional object as a collection of points, generates a reference relationship of the attribute information between hierarchies, and, in the generated hierarchy of the attribute information, organizes the attribute information of the hierarchy into sub-hierarchies and generates a reference relationship of the attribute information between the sub-hierarchies.
[0008] An information processing device according to another aspect of the present technology is an information processing device including: a hierarchical structure unit that hierarchically structures attribute information of a point cloud that expresses an object having a three-dimensional shape as a collection of points, and generates a reference relationship of the attribute information between hierarchies; a sub-hierarchical structure unit that sub-hierarchically structures the attribute information of the hierarchy of the attribute information generated by the hierarchical structure unit, and generates a reference relationship of the attribute information between sub-hierarchies; and an inverse-hierarchical structure unit that inversely hierarchically structures the attribute information based on the reference relationship of the attribute information between the hierarchies generated by the hierarchical structure unit and the reference relationship of the attribute information between the sub-hierarchies generated by the sub-hierarchical structure unit.
[0009] Another aspect of the information processing method of the present technology is an information processing method of hierarchizing attribute information of a point cloud that represents an object of a three-dimensional shape as a collection of points, generating a reference relationship of the attribute information between hierarchies, sub-hierarchizing the attribute information of the hierarchy of the generated attribute information, generating a reference relationship of the attribute information between sub-hierarchies, and inverting the hierarchical relationship of the attribute information based on the reference relationship of the attribute information between the generated hierarchies and the reference relationship of the attribute information between the generated sub-hierarchies.
[0010] In an information processing device and method according to one aspect of the present technology, attribute information of a point cloud that represents a three-dimensional object as a collection of points is hierarchically organized, and a reference relationship of attribute information between the hierarchies is generated. In the generated hierarchy of attribute information, the attribute information of the hierarchy is sub-hierarchically organized, and a reference relationship of attribute information between the sub-hierarchies is generated.
[0011] In an information processing device and method according to another aspect of the present technology, attribute information of a point cloud that represents a three-dimensional object as a collection of points is hierarchically organized, and a reference relationship of attribute information between the hierarchies is generated, and in the generated hierarchy of attribute information, the attribute information of the hierarchy is sub-hierarchically organized, and a reference relationship of attribute information between the sub-hierarchies is generated, and the attribute information is inversely hierarchically organized based on the reference relationship of attribute information between the generated hierarchies and the reference relationship of attribute information between the generated sub-hierarchies. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a reference relationship of attribute information according to the related art. [Diagram 2] FIG. 13 is a diagram illustrating an example of a reference relationship according to Morton's order in a hierarchy. [Diagram 3] FIG. 1 is a diagram for explaining a hierarchical method and an inverse hierarchical method for attribute data. [Figure 4] FIG. 11 is a diagram illustrating an example of how Sub LoD is generated. [Diagram 5] FIG. 11 is a diagram illustrating an example of how Sub LoD is generated. [Figure 6] FIG. 13 is a diagram illustrating an example of a reference relationship. [Figure 7] FIG. 13 is a diagram illustrating an example of a reference relationship. [Figure 8] FIG. 2 is a block diagram showing an example of the main configuration of an encoding device. [Figure 9] 13 is a block diagram showing an example of the main configuration of an attribute information encoding unit. [Figure 10] 2 is a block diagram showing an example of a main configuration of a hierarchical processing unit. FIG. [Figure 11] 11 is a flowchart illustrating an example of the flow of an encoding process. [Figure 12] 13 is a flowchart illustrating an example of the flow of an attribute information encoding process. [Figure 13] 13 is a flowchart illustrating an example of the flow of a hierarchical process. [Figure 14] FIG. 2 is a block diagram showing an example of the main configuration of a decoding device. [Figure 15] 13 is a block diagram showing an example of the main configuration of an attribute information decoding unit. FIG. [Figure 16] 2 is a block diagram showing an example of the main configuration of an inverse layer processing unit. FIG. [Figure 17] 13 is a flowchart illustrating an example of the flow of a decoding process. [Figure 18] 13 is a flowchart illustrating an example of the flow of an attribute information decoding process. [Figure 19] 13 is a flowchart illustrating an example of the flow of a layer inversion process. [Figure 20] FIG. 1 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1.Generating Sub LoD 2. First embodiment (encoding device) 3. Second embodiment (decoding device) 4. Notes
[0014] <1. Generation of Sub LoD> <References supporting technical content and technical terms> The scope of disclosure of the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent documents that were publicly known at the time of filing the application.
[0015] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: (mentioned above)
[0016] In other words, the contents described in the above-mentioned non-patent documents also serve as a basis for determining support requirements.
[0017] <Point cloud> Previously, there was 3D data such as point clouds, which represent three-dimensional structures based on point group position information and attribute information, and meshes, which are composed of vertices, edges, and faces and define three-dimensional shapes using polygonal representations.
[0018] For example, in the case of a point cloud, a three-dimensional structure (an object having a three-dimensional shape) is represented as a collection of many points (a point cloud). In other words, the data of the point cloud (also referred to as point cloud data) is composed of geometry data (also referred to as position information) and attribute data (also referred to as attribute information) of each point of the point cloud. The attribute data can include any information. For example, color information, reflectance information, normal information, etc. may be included in the attribute data. Therefore, the data structure is relatively simple, and any three-dimensional structure can be represented with sufficient accuracy by using a sufficiently large number of points.
[0019] <Quantization of position information using voxels> Since such point cloud data has a relatively large amount of data, an encoding method using voxels was devised to compress the amount of data by encoding, etc. A voxel is a three-dimensional region for quantizing geometry data (position information).
[0020] That is, the three-dimensional area containing the point cloud is divided into small three-dimensional areas called voxels, and each voxel indicates whether it contains a point or not. In this way, the position of each point is quantized in voxel units. Therefore, by converting point cloud data into such voxel data (also called voxel data), it is possible to suppress an increase in the amount of information (typically, to reduce the amount of information).
[0021] <octree> Furthermore, for geometry data, it was considered to construct an octree using such voxel data. An octree is a tree-structured version of voxel data. The value of each bit in the lowest node of this octree indicates whether or not each voxel has a point. For example, a value of "1" indicates a voxel that contains a point, and a value of "0" indicates a voxel that does not contain a point. In an octree, one node corresponds to eight voxels. In other words, each node in the octree is made up of eight bits of data, and these eight bits indicate whether or not each of the eight voxels has a point.
[0022] The upper nodes of the Octree indicate whether or not there is a point in the area that combines the eight voxels corresponding to the lower nodes belonging to that node. In other words, the upper nodes are generated by combining the information of the voxels of the lower nodes. Note that a node with a value of "0", that is, a node in which none of the corresponding eight voxels contains a point, is deleted.
[0023] In this way, a tree structure (Octree) consisting of nodes whose values are not "0" is constructed. In other words, the Octree can indicate the presence or absence of voxel points at each resolution. By encoding the Octree, the position information can be decoded from the highest resolution (top layer) to the desired layer (resolution) to restore the point cloud data at that resolution. In other words, it is possible to easily decode at any resolution without decoding information at unnecessary layers (resolutions). In other words, it is possible to achieve scalability of voxels (resolutions).
[0024] In addition, by omitting nodes with a value of "0" as described above, it is possible to reduce the resolution of voxels in areas where no points exist, thereby further suppressing the increase in the amount of information (typically reducing the amount of information).
[0025] <lifting> On the other hand, when encoding attribute data (attribute information), the geometry data (position information) is assumed to be known, including degradation due to encoding, and encoding is performed using the positional relationship between points. As a method for encoding such attribute data, a method using a transformation called RAHT (Region Adaptive Hierarchical Transform) or Lifting as described in Non-Patent Document 2 has been considered. By applying these technologies, it is also possible to hierarchically organize attribute data, like the Octree of geometry data.
[0026] For example, in the case of Lifting described in Non-Patent Document 2, attribute data of each point is encoded as a difference value with a predicted value derived using attribute data of other points. At that time, each point is hierarchically organized, and the difference value is derived according to the hierarchical structure.
[0027] That is, for the attribute data of each point, each point is classified into a prediction point and a reference point, a prediction value of the attribute data of the prediction point is derived using the attribute data of the reference point, and a difference value between the attribute data of the prediction point and the prediction value is derived. By recursively repeating such processing for the reference points, the attribute data of each point is hierarchically organized.
[0028] However, this hierarchical structure is generated independently of the hierarchical structure of the geometry data (e.g., Octree), and does not basically correspond to the hierarchical structure of the geometry data. In order to restore the point cloud data, it is necessary to make the geometry data correspond to the attribute data, and for that purpose, it is necessary to decode the geometry data and the attribute data up to the highest resolution (i.e., the lowest layer). In other words, the method to which Lifting described in Non-Patent Document 2 is applied does not support resolution scalable decoding.
[0029] <Hierarchical structure for scalable decryption> In contrast, the hierarchical structure described in Non-Patent Document 3 supports resolution scalable decoding. In the method described in Non-Patent Document 3, attribute data is hierarchical structured to match the hierarchical structure of the octree of the geometry data. In other words, when a point exists in an area corresponding to a voxel of the geometry data (attribute data corresponding to the point exists), a reference point and a predicted point are selected so that a point also exists in a voxel one level above the voxel (attribute data corresponding to the point exists). In other words, attribute information is hierarchical structured according to the hierarchical structure of the octree of the geometry data.
[0030] By associating the hierarchical structure of the attribute data with the hierarchical structure of the geometry data, it is possible to easily restore point cloud data of a desired resolution without decoding down to the lowest layer. In this way, the method applying the technology described in Non-Patent Document 3 supports resolution scalable decoding.
[0031] <Reference relationship within the hierarchy> In the above-mentioned method, a reference relationship of attribute information is generated between layers, which means that attribute information in a layer to be processed can be predicted by referring to attribute information in other layers.
[0032] In contrast, in the case of the technique described in Non-Patent Document 4, attribute information in the hierarchical layer to be processed can be referenced if it has already been decoded. For example, the hierarchical attribute data (Attribute Lod) 1 shown in A of Fig. 1 is obtained by classifying and hierarchizing attribute data 2 for each point as shown in B of Fig. 1. In B of Fig. 1, the vertical direction indicates the layer (LoD (Level Of Detail)), and each circle indicates attribute data 2 for each point. In B of Fig. 1, only one circle is given a code, but all the circles shown in B of Fig. 1 are attribute data 2 for each point.
[0033] Decoding is performed sequentially, layer by layer, starting from the top. Therefore, when decoding attribute data 2A for a certain point, attribute data 2 for each point in the top layer, indicated by a white circle, has already been decoded, so it is possible to refer to them.
[0034] In addition, within a layer, the attribute data 2 for each point is sorted in Morton order and decoded in that order. In B of Fig. 1, the attribute data 2 for each point in each layer is sorted in Morton order. In the figure, the horizontal direction from left to right corresponds to Morton order.
[0035] In other words, when the attribute data 2A of a certain point is decoded, the attribute data 2B of each point located to the left of it has already been decoded and can be referenced.
[0036] By sorting in Morton order, the attribute data 2 for each point is scanned for each three-dimensional partial region obtained by dividing the three-dimensional space. Therefore, a directionality (bias) of the three-dimensional spatial position occurs in the processing order.
[0037] For example, each point of the point cloud data is assumed to exist within the three-dimensional region 3 in FIG. 2. As described above, in B in FIG. 1, the decoded attribute data 2B for each point is located before the attribute data 2A for a certain point in the Morton order. Therefore, in the three-dimensional region 3 in FIG. 2, the decoded attribute data 2B for each point shown in B in FIG. 1 exists within a partial region 3B located in a predetermined direction as viewed from the attribute data 2A for a certain point. In other words, a bias occurs in the relative position (direction) of the decoded attribute data 2B for each point when the position of the attribute data 2A for a certain point is used as a reference.
[0038] By performing processing according to the Morton order, such bias occurs in the reference direction during prediction, and this bias may reduce prediction accuracy and coding efficiency.
[0039] <Sub-hierarchy> Therefore, for example, as described in "Method 1" in the top row of the table in Figure 3, in hierarchizing attribute data (attribute information), a sub-hierarchy (Sub LoD) is further generated within the hierarchy (LoD), and attribute data reference relationships are generated between the sub-hierarchies (i.e., attribute data reference relationships within the hierarchy (LoD) are generated).
[0040] For example, attribute data (attribute information) of a point cloud that represents a three-dimensional object as a collection of points is used to generate attribute data reference relationships between hierarchies, and then, within the generated attribute data hierarchy, the attribute data of that hierarchy is divided into sub-hierarchies, and attribute data reference relationships between the sub-hierarchies are generated.
[0041] For example, an information processing device may include a hierarchical structure unit that hierarchically structures attribute data (attribute information) of a point cloud that represents a three-dimensional object as a collection of points and generates a reference relationship of attribute data between the hierarchies, and a sub-hierarchical structure unit that sub-hierarchically structures the attribute data of the hierarchy of the attribute data generated by the hierarchical structure unit and generates a reference relationship of attribute data between the sub-hierarchies.
[0042] In this way, it is possible to generate a reference relationship of attribute data within a hierarchy so as to suppress bias in the prediction direction, and it is possible to suppress a decrease in coding efficiency.
[0043] In this case, the attribute data may be layered in any manner. For example, Lifting, which is not compatible with scalable decoding and is described in Non-Patent Document 2, may be used. In addition, a layering technique compatible with scalable decoding and described in Non-Patent Document 3 may be applied. Of course, other methods may be used.
[0044] The method of creating a sub-hierarchy may also be arbitrary. For example, as described in "Method 1-1" in the second row from the top of the table shown in Fig. 3, a process of sampling (selecting) a part of the attribute data for each point in a hierarchy to create a sub-hierarchy may be repeated recursively. In other words, a sub-hierarchy may be generated by sampling a part from a group of nodes in a hierarchy (LoD) to be processed in the hierarchical structure of the attribute data.
[0045] For example, as described in "Method 1-1-1" in the third row from the top of the table in Figure 3, sub-hierarchies may be generated by arranging the attribute data of the hierarchy to be processed in Morton order and sampling a portion of it at equal intervals.
[0046] For example, as in the case of B in FIG. 1, for hierarchical attribute data, the attribute data for each point of the hierarchy to be sub-hierarchicalized is sorted in Morton order. Then, the sorted attribute data for each point is sampled at equal intervals. FIG. 4 shows an example in which every third, that is, every fourth, is sampled (LodUniformQuant = 4). LodUniformQuant is a syntax element indicating the sampling interval. LodUniformQuant = 4 indicates that every fourth is sampled from the column of attribute data for each point sorted in Morton order. In other words, when one attribute data is sampled, the fourth attribute data in the Morton order is sampled from there.
[0047] In A of Fig. 4, the point-specific attribute data 12 indicated by circles are arranged in Morton order, and one out of every four (point-specific attribute data 12 indicated by gray circles) is sampled. In other words, as shown in B of Fig. 4, the lowest sub-hierarchy (Sub LoD) consisting of point-specific attribute data 12 indicated by white circles that have not been sampled is generated.
[0048] Similarly, one out of every four (point attribute data 12 indicated by black circles) is sampled from the row of sampled point attribute data 12. That is, as shown in Fig. 4C, the second-lowest sub-level (Sub LoD) is generated, which is made up of point attribute data 12 indicated by gray circles that have not been sampled.
[0049] Similarly, one out of every four (attribute data 12 for each point indicated by a circle with a diagonal line) is sampled from the row of sampled attribute data 12 for each point. That is, as shown in D of Fig. 4, a sub-hierarchy third from the bottom is generated, which is made up of attribute data 12 for each point indicated by a black circle that has not been sampled.
[0050] In this case, since the attribute data 12 for each point is sampled, a top-level sub-hierarchy made up of the attribute data 12 for each sampled point is also generated (D in FIG. 4).
[0051] A reference relationship is also created between such sub-hierarchies, that is, attribute data of a higher sub-hierarchy is decoded first, making it possible to refer to the attribute data of the higher sub-hierarchy.
[0052] In this case, since the entire string of attribute data 12 for each point is sampled to generate a sub-hierarchy, there is less bias in the positions of those points in three-dimensional space. In other words, compared to the method described in Non-Patent Document 4, it is possible to suppress bias in the reference direction of attribute data. Therefore, it is possible to suppress a decrease in prediction accuracy and a decrease in coding efficiency.
[0053] In this case, the attribute data 12 for each point is also sorted in Morton order, so that it is possible to refer to relatively nearby attribute data in three-dimensional space. Therefore, it is possible to suppress the decrease in prediction accuracy and the decrease in coding efficiency more than when the attribute data is selected randomly.
[0054] In addition, in addition to the attribute data in the higher sub-hierarchy, it may be possible to refer to the decoded attribute data in the same sub-hierarchy as the attribute data to be processed.
[0055] 5 shows attribute data 12 for each point in a certain hierarchy arranged in the order of decoding. In this case, if attribute data 12A for a certain point is to be decoded, attribute data 12 for each point in the range indicated by the double-headed arrow 12B (i.e., attribute data 12 for each point located to the left of attribute data 12A for a certain point) has already been decoded and can be referenced. In this way, the number of candidates for reference points increases, making it possible to suppress a decrease in prediction accuracy and a decrease in coding efficiency.
[0056] In the above, it has been described that the attribute data is sampled at equal intervals when generating the sub-hierarchy, but this is not limiting and the data may be sampled at non-equidistant intervals. In other words, as described in "Method 1-1-2" in the fourth row from the top of the table shown in Fig. 3, the attribute data of the processing target hierarchy may be arranged in Morton order, and a portion of the data may be sampled at non-equidistant intervals to generate the sub-hierarchy.
[0057] For example, attribute data in a sub-hierarchy may be sampled at non-uniform intervals (i.e. not at regular intervals), for example alternating between every third and every second (LodUniformQuant = 2or3).
[0058] Also, for example, the sampling interval may be changed for each sub-layer. For example, the lowest sub-layer may sample every third sub-layer (LodUniformQuant = 2), and higher sub-layers may sample every fourth sub-layer (LodUniformQuant = 4). In other words, the sampling interval may be changed according to the depth of the sub-layer.
[0059] In this way, for example, the sampling interval can be set according to the resolution, and for example, degradation of image quality and degradation of coding efficiency can be suppressed.
[0060] <weight value> Such sub-hierarchy may be reflected in the weight value of the attribute data. In the case of conventional methods, the attribute data for each point is weighted based on the reference relationship between the hierarchies (LoDs) (e.g., the distance of reference, the number of references, etc.). When sub-hierarchy is performed as described above, the reference relationship between the sub-hierarchies (e.g., the distance of reference, the number of references, etc.) may also be reflected in the weight value. In other words, as described in "Method 1-2" in the fifth row from the top of the table shown in FIG. 3, the weight value may be derived using the reference relationship between the sub-hierarchies (Sub LoDs).
[0061] For example, when attribute data is layered (LoD) using Lifting described in Non-Patent Document 2, a weight value is set for each attribute data based on the reference relationship. When attribute data layered by the method of setting weight values in this way is sub-layered as described above, the reference relationship between sub-layers is reflected in the weight value set for each attribute data. That is, the weight value of each attribute data is derived using the reference relationship between layers and between sub-layers constructed as shown by the arrows in FIG. 6. In this way, the reference relationship between sub-layers can be reflected in the weight value between layers generated by a method that does not support scalable decoding.
[0062] For example, when attribute data is layered (LoD) using a method corresponding to scalable decoding described in Non-Patent Document 3, a fixed weight value (weight value for each layer) is assigned to each attribute data for each layer (LoD). When attribute data layered using the method for setting weight values in this way is sub-layered as described above, the weight value for each layer reflects the reference relationship between the sub-layers, as described in "Method 1-2-1" in the sixth row from the top of the table shown in Fig. 3. In other words, the weight value for each attribute data is derived by updating the weight value for each layer using the reference relationship between the sub-layers as shown by the arrows in A of Fig. 7.
[0063] For example, a coefficient α may be set according to the number of references between sub-hierarchies, and the weight value w1 for each hierarchy may be updated by multiplying the coefficient α by the weight value w1 for each hierarchy, as shown in B of FIG.
[0064] Furthermore, when updating the weight value w1 of a certain layer, the weight value w2 of another layer may be updated so as to correspond to the update of the weight value w1. For example, as shown in C of Fig. 7, when multiplying the weight value w1 by the coefficient α, the weight value w2 may be multiplied by (1-α).
[0065] In this way, the reference relationship between sub-layers can be reflected in the weighting values between layers generated by a method that does not support scalable decoding, thereby making it possible to further suppress a decrease in coding efficiency.
[0066] <Signaling of control information> As described in "Method 1-3" in the seventh row from the top of the table shown in Fig. 3, control information regarding the sub-hierarchical division of such attribute data may be signaled. The content of this control may be any type.
[0067] <sub_lod_enable_flag> For example, control information indicating whether or not sub-layering of attribute data is permitted may be signaled (transmitted from the encoding side to the decoding side). The control information indicating whether or not sub-layering is permitted is flag information (e.g., sub_lod_enable_flag) indicating whether or not sub-layering can be performed in the target data unit. For example, when this sub_lod_enable_flag is true (e.g., "1"), it indicates that sub-layering can be performed. Also, when sub_lod_enable_flag is false (e.g., "0"), it indicates that sub-layering cannot be performed.
[0068] If the sub_lod_enable_flag is not signaled (transmitted), the decoding side may consider the sub_lod_enable_flag to be false. Conversely, if the sub_lod_enable_flag is not signaled (transmitted), the decoding side may consider the sub_lod_enable_flag to be true.
[0069] When sub_lod_enable_flag is false, since sub-layering is not permitted, signaling of other control information related to sub-layering for that data unit can be omitted. Therefore, it is possible to suppress a decrease in coding efficiency. In other words, on the decoding side, parsing of other control information related to sub-layering for that data unit can be omitted. That is, on the decoding side, it is only necessary to parse other control information related to sub-layering for only data units for which sub_lod_enable_flag is true. Therefore, it is possible to suppress an increase in the load of the decoding process.
[0070] As described above, the control information indicating whether or not sub-layering is permitted can also be considered as control information indicating the presence or absence of signaling of other control information related to sub-layering for the data unit. Also, the control information indicating whether or not sub-layering is permitted can also be considered as control information indicating whether or not sub-layering is prohibited.
[0071] The control information indicating whether or not the sub-layering is permitted may be targeted at any data unit, and may be signaled for each data unit. For example, one piece of attribute data may be targeted as the data unit, and sub_lod_enable_flag may be signaled for each piece of attribute data. Also, for example, multiple sub_lod_enable_flag may be transmitted together for each data unit (for example, for each sequence) higher than the attribute data.
[0072] <sub_lod_distance> For example, control information for controlling the sampling interval (or sampling period) during sub-hierarchy generation may be signaled. The control information for controlling the sampling interval is information (e.g., sub_lod_distance) indicating the interval at which sampling should be performed when sub-hierarchy generation is performed for the target data unit. For example, when sub_lod_distance = 2, this indicates that the sampling period is 2, i.e., one out of every two nodes is sampled (sampled every other node). In other words, when sub_lod_distance > 2, sampling is performed at equal intervals indicated by sub_lod_distance, and a sub-hierarchy is generated.
[0073] Note that sub_lod_distance = 0 indicates that no sub-layering is performed (no sub-layer (Sub LoD) is formed. In other words, sub_lod_distance can be said to be control information that indicates whether or not to perform sub-layering.
[0074] Also, if this sub_lod_distance is not signaled (transmitted), it may be considered that sub_lod_distance=0 on the decoding side.
[0075] Furthermore, the sampling intervals may be non-uniform. For example, multiple sub_lod_distances may be signaled for the same data units, and multiple intervals (cycles) may be assigned. For example, the sub_lod_distance values may be set to "2" and "3", and these intervals may be adopted in sequence, such as 2→3→2→3, to realize non-uniform intervals. Also, the sampling intervals may be represented by a function, a table identification number, or the like.
[0076] The control information for controlling the sampling interval during this sub-hierarchical division can target any data unit, and may be signaled for each arbitrary data unit. For example, the attribute data hierarchy (LoD) may be the target data unit, and sub_lod_distance may be signaled for each hierarchy of the attribute data.
[0077] <sub_lod_mode> For example, control information (eg, sub_lod_mode) indicating the sampling method for sub-layering may be signaled.
[0078] For example, sub_lod_mode = 0 indicates that sampling is performed at equal intervals in all sub-hierarchies, i.e., sampling is performed at one type of interval in all sub-hierarchies.
[0079] Furthermore, sub_lod_mode = 1 indicates that the sampling intervals are pattern 1 (non-uniform intervals). Similarly, sub_lod_mode = 2 indicates that the sampling intervals are pattern 2 (non-uniform intervals) which is different from pattern 1.
[0080] By making the sampling intervals non-uniform, it is possible to change the sampling interval depending on the depth of the sub-level (SubLod), for example. It is also possible to change the sampling interval within a sub-level. In other words, the above-mentioned Pattern 1 and Pattern 2 show the sampling interval patterns for all sub-levels. The pattern distance can be overridden by the header, etc.
[0081] <Other> Note that the control information regarding the sub-layering is not limited to the above example. In addition, a plurality of pieces of control information may be applied. For example, both sub_lod_distance and sub_lod_mode may be applied. In addition, for example, sub_lod_enable_flag may be applied together with sub_lod_distance or sub_lod_mode, or both.
[0082] <Reverse hierarchy> The attribute data that has been organized into hierarchical and sub-hierarchical layers as described above may be inverted into a hierarchical layer by utilizing the reference relationships between the sub-layers (Sub LoDs), as described in "Method 2" in the eighth row from the top of the table in Figure 3.
[0083] Also, for example, as described in "Method 2-1" in the ninth row from the top of the table shown in Figure 3, a sub-hierarchy (Sub LoD) may be generated in the same manner as in "Method 1" described above, and the hierarchical inversion may be performed using the reference relationships between the generated sub-hierarchies.
[0084] For example, the attribute data (attribute information) of a point cloud that represents a three-dimensional object as a collection of points may be hierarchically organized, attribute data reference relationships between the hierarchies may be generated, the attribute data may be sub-hierarchically organized within the hierarchy, attribute data reference relationships between the sub-hierarchies may be generated, and the attribute data may be inversely hierarchically organized based on the generated attribute data reference relationships between the hierarchies and the generated attribute data reference relationships between the sub-hierarchies.
[0085] For example, an information processing device may include a hierarchical structure forming unit that hierarchically structures attribute data (attribute information) of a point cloud that represents an object having a three-dimensional shape as a collection of points and generates a reference relationship of attribute data between the hierarchies, a sub-hierarchical structure forming unit that sub-hierarchically structures the attribute data within the hierarchy generated by the hierarchical structure forming unit and generates a reference relationship of attribute data between the sub-hierarchies, and an inverse-hierarchical structure forming unit that inversely hierarchically structures the attribute data based on the reference relationship of attribute data between the hierarchies generated by the hierarchical structure forming unit and the reference relationship of attribute data between the sub-hierarchies generated by the sub-hierarchical structure forming unit.
[0086] In this way, attribute data that has been hierarchically organized or sub-hierarchically organized by applying the above-mentioned "Method 1" or the like can be correctly dehierarchically organized, thereby suppressing a decrease in coding efficiency.
[0087] In forming the sub-hierarchy, for example, the "Method 1-1-1" in the third row from the top in the table shown in Fig. 3 may be applied. In other words, the attribute data of the hierarchy to be processed may be arranged in Morton's order, and a portion of the data may be sampled at equal intervals to generate the sub-hierarchy.
[0088] In addition, in forming the sub-hierarchy, for example, "Method 1-1-2" in the fourth row from the top in the table shown in Fig. 3 may be applied. In other words, the attribute data of the processing target hierarchy may be arranged in Morton order, and a part of the data may be sampled at non-uniform intervals to generate the sub-hierarchy.
[0089] In addition, in the sub-hierarchy, for example, as described in "Method 2-2" in the 10th row from the top of the table shown in Figure 3, a sub-hierarchy (Sub LoD) may be generated based on control information regarding the sub-hierarchy of attribute data signaled as in the above-mentioned "Method 1-3", and inverse hierarchical structure may be performed using the reference relationships between the generated sub-hierarchies.
[0090] For example, the attribute data may be divided into sub-hierarchies based on the control information (eg, sub_lod_enable_flag) indicating whether or not the above-mentioned sub-hierarchy of the attribute data is permitted.
[0091] Furthermore, for example, the attribute data may be divided into sub-hierarchies based on control information (for example, sub_lod_distance) indicating the sampling interval of the attribute data described above.
[0092] Furthermore, for example, the attribute data may be divided into sub-hierarchies based on control information (for example, sub_lod_mode) indicating the method of sampling the attribute data described above.
[0093] Of course, the attribute data may be divided into sub-hierarchies based on control information other than the above, or may be divided into sub-hierarchies based on a plurality of control information.
[0094] By performing the inverse hierarchical structure as described above, the sub-hierarchical attribute data can be inversely hierarchical structured more accurately. Therefore, bias in the prediction direction can be suppressed, and a decrease in coding efficiency can be suppressed.
[0095] <2. First embodiment> <Encoding device> Next, a device to which the present technology described above in <1. Generation of Sub LoD> is applied will be described. FIG. 8 is a block diagram showing an example of a configuration of an encoding device, which is one aspect of an information processing device to which the present technology is applied. The encoding device 100 shown in FIG. 8 is a device that encodes a point cloud (3D data). The encoding device 100 encodes the point cloud by applying the present technology described above in <1. Generation of Sub LoD>.
[0096] Note that Fig. 8 shows the main processing units, data flows, etc., and is not limited to all that is shown in Fig. 8. In other words, in encoding device 100, there may be processing units that are not shown as blocks in Fig. 8, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 8.
[0097] As shown in FIG. 8, the encoding device 100 includes a position information encoding unit 101, a position information decoding unit 102, a point cloud generating unit 103, an attribute information encoding unit 104, and a bit stream generating unit 105.
[0098] The position information encoding unit 101 encodes geometry data (position information) of a point cloud (3D data) input to the encoding device 100. Any encoding method may be used as long as it is compatible with scalable decoding. For example, the position information encoding unit 101 hierarchically organizes the geometry data to generate an octree, and encodes the octree. In addition, for example, filtering and quantization for noise suppression (denoising) may be performed. The position information encoding unit 101 supplies encoded data of the generated geometry data to the position information decoding unit 102 and the bit stream generating unit 105.
[0099] The position information decoding unit 102 obtains the encoded data of the geometry data supplied from the position information encoding unit 101, and decodes the encoded data. Any decoding method may be used as long as it is compatible with the encoding by the position information encoding unit 101. For example, processing such as filtering and inverse quantization for denoising may be performed. The position information decoding unit 102 supplies the generated geometry data (decoded result) to the point cloud generating unit 103.
[0100] The point cloud generating unit 103 acquires attribute data (attribute information) of the point cloud input to the encoding device 100 and geometry data (decoded result) supplied from the position information decoding unit 102. The point cloud generating unit 103 performs processing (recolor processing) to match the attribute data with the geometry data (decoded result). The point cloud generating unit 103 supplies the attribute data associated with the geometry data (decoded result) to the attribute information encoding unit 104.
[0101] The attribute information encoding unit 104 acquires the geometry data (decoded result) and attribute data supplied from the point cloud generating unit 103. The attribute information encoding unit 104 encodes the attribute data using the geometry data (decoded result) to generate encoded data of the attribute data.
[0102] At this time, the attribute information encoding unit 104 applies the present technology described above in <1. Generation of Sub LoD> to encode the attribute data. For example, in hierarchizing the attribute data (attribute information), the attribute information encoding unit 104 generates a sub-hierarchy (Sub LoD) within a hierarchy (LoD) and generates a reference relationship of attribute data between the sub-hierarchies. The attribute information encoding unit 104 supplies the generated encoded data of the attribute data to the bitstream generating unit 105.
[0103] The bitstream generating unit 105 obtains the coded data of the geometry data supplied from the position information encoding unit 101. The bitstream generating unit 105 also obtains the coded data of the attribute data supplied from the attribute information encoding unit 104. The bitstream generating unit 105 generates a bitstream including these coded data. The bitstream generating unit 105 outputs the generated bitstream to the outside of the encoding device 100.
[0104] With this configuration, the encoding device 100 can generate sub-hierarchies within a hierarchy, generate reference relationships of attribute data between the sub-hierarchies, and encode the attribute data using the reference relationships. This makes it possible to generate reference relationships of attribute data within a hierarchy so as to suppress bias in the prediction direction, and suppress a decrease in encoding efficiency.
[0105] Each of these processing units (position information encoding unit 101 to bit stream generating unit 105) of the encoding device 100 has an arbitrary configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0106] <Attribute information encoding section> Fig. 9 is a block diagram showing a main configuration example of the attribute information encoding unit 104 (Fig. 8). Note that Fig. 9 shows main processing units, data flows, etc., and is not limited to all that are shown in Fig. 9. In other words, in the attribute information encoding unit 104, there may be processing units not shown as blocks in Fig. 9, and there may be processing and data flows not shown as arrows, etc. in Fig. 9.
[0107] As shown in FIG. 8, the attribute information encoding unit 104 includes a hierarchical processing unit 111, a quantization unit 112, and an encoding unit 113.
[0108] The layering processing unit 111 performs processing related to layering of attribute data. For example, the layering processing unit 111 acquires attribute data and geometry data (decoded result) supplied from the point cloud generating unit 103. The layering processing unit 111 layered the attribute data using the geometry data. At that time, the layering processing unit 111 performs layering by applying the present technology described above in <1. Generation of Sub LoD>. For example, in layering the attribute data (attribute information), the layering processing unit 111 further generates a sub-layer (Sub LoD) within the layer (LoD) and generates a reference relationship of attribute data between the sub-layers. The layering processing unit 111 supplies the layered attribute data (difference value) to the quantizing unit 112.
[0109] At this time, the layering processing unit 111 can also generate control information related to sub-layering. In this case, the layering processing unit 111 supplies the generated control information to the quantization unit 112 together with the attribute data (difference value).
[0110] The quantization unit 112 acquires the attribute data supplied from the hierarchical processing unit 111 (and the control information if the control information is generated in the hierarchical processing unit 111). The quantization unit 112 quantizes the attribute data. Any quantization method may be used. The quantization unit 112 supplies the quantized attribute data (and the control information) to the encoding unit 113.
[0111] The encoding unit 113 obtains the quantized attribute data (and control information) supplied from the quantization unit 112. The encoding unit 113 encodes the attribute data to generate encoded data of the attribute data. Any encoding method may be used. Furthermore, when control information related to sub-hierarchization is supplied, the encoding unit 113 includes the control information in the generated encoded data. In other words, the encoding unit 113 generates encoded data of the attribute data including the control information related to sub-hierarchization. The encoding unit 113 supplies the generated encoded data to the bit stream generation unit 105.
[0112] By performing hierarchical division as described above, the attribute information encoding unit 104 can generate sub-hierarchies within a hierarchy, generate reference relationships of attribute data between the sub-hierarchies, and encode the attribute data using the reference relationships. This makes it possible to generate reference relationships of attribute data within a hierarchy so as to suppress bias in the prediction direction, and suppress a decrease in encoding efficiency.
[0113] In addition, these processing units (layering processing unit 111 to encoding unit 113) have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0114] <Hierarchical Processing Unit> Fig. 10 is a block diagram showing an example of the main configuration of the hierarchical processing unit 111 (Fig. 9). Note that Fig. 10 shows the main processing units, data flows, etc., and is not limited to all that are shown in Fig. 10. In other words, in the hierarchical processing unit 111, there may be processing units that are not shown as blocks in Fig. 10, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 10.
[0115] As shown in FIG. 10, the layering processing unit 111 includes a control unit 121, a layering processing unit 122, a Sub LoD generating unit 123, an inversion unit 124, and a weighting unit 125.
[0116] The control unit 121 performs processing related to the control of hierarchical and sub-hierarchical formation. For example, the control unit 121 acquires attribute data and geometry data (decoded results) supplied from the point cloud generating unit 103. The control unit 121 supplies the acquired attribute data and geometry data (decoded results) to the hierarchical processing unit 122.
[0117] Furthermore, the control unit 121 controls the layering processing unit 122 and the Sub LoD generating unit 123 to execute layering and sub-layering. For example, the control unit 121 applies the present technology described above in <1. Sub LoD> to execute layering and sub-layering. For example, the control unit 121 executes sub-layering for a desired layer. That is, the control unit 121 can execute sub-layering for some layers, or execute sub-layering for all layers.
[0118] The control unit 121 can also generate control information related to the sub-hierarchical division of the attribute data. For example, the control unit 121 can generate various syntax elements (e.g., sub_lod_enable_flag, sub_lod_distance, sub_lod_mode, etc.) described above in <1. Generation of Sub LoD> as control information. The control unit 121 supplies the control information thus generated to the quantization unit 112, which transmits the control information to the decoding side.
[0119] The layering processing unit 122 performs processing related to layering of attribute data. For example, the layering processing unit 122 obtains attribute data and geometry data (decoding results) supplied from the control unit 121.
[0120] The layering processing unit 122 layers the acquired attribute data using the acquired geometry data under the control of the control unit 121. Any layering method may be used. For example, a method not compatible with scalable decoding such as Lifting described in Non-Patent Document 2 may be used, or a method compatible with scalable decoding such as that described in Non-Patent Document 3 may be used. The layering processing unit 122 supplies the layered attribute data and geometry data to the Sub LoD generating unit 123.
[0121] The Sub LoD generating unit 123 performs processing related to sub-hierarchization. For example, the Sub LoD generating unit 123 obtains hierarchical attribute data and geometry data supplied from the hierarchical processing unit 122.
[0122] The Sub LoD generation unit 123 generates a sub-hierarchy within the hierarchy of attribute data in accordance with the control of the control unit 121. The Sub LoD generation unit 123 performs such sub-hierarchy generation by using geometry data. That is, the Sub LoD generation unit 123 generates a reference relationship of attribute data within a hierarchy (LoD) as in "Method 1" in the top row of the table shown in Fig. 3 described above in <1. Generation of Sub LoD>.
[0123] In this way, the Sub LoD generation unit 123 can generate a reference relationship of attribute data within a hierarchy so as to suppress bias in the prediction direction, and can suppress a decrease in coding efficiency.
[0124] This generation method is arbitrary. For example, the Sub LoD generation unit 123 may recursively repeat a process of sampling (selecting) a part of the attribute data for each point in a hierarchy to create a sub-hierarchy, as in "Method 1-1" in the second row from the top of the table shown in Fig. 3.
[0125] In addition, the Sub LoD generation unit 123 may generate a sub-hierarchy by arranging the attribute data of the hierarchy to be processed in Morton order and sampling a portion of it at equal intervals, as in "Method 1-1-1" in the third row from the top of the table shown in Figure 3.
[0126] In addition, the Sub LoD generation unit 123 may generate a sub-hierarchy by arranging the attribute data of the hierarchy to be processed in Morton order and sampling some of it at non-equidistant intervals, as in "Method 1-1-2" in the fourth row from the top of the table shown in Figure 3.
[0127] Of course, methods other than these may be used. If sub-layering is not permitted by the control unit 121, the Sub LoD generation unit 123 can omit sub-layering. In other words, the Sub LoD generation unit 123 can perform sub-layering only on layers permitted by the control unit 121.
[0128] The Sub LoD generating unit 123 supplies the attribute data thus sub-hierarchized under the control of the control unit 121 to the inverting unit 124 .
[0129] The inversion unit 124 performs processing related to layer inversion. For example, the inversion unit 124 obtains attribute data supplied from the Sub LoD generation unit 123. In this attribute data, information on each layer is hierarchically organized in the order in which it was generated.
[0130] The inversion unit 124 inverts the hierarchical levels of the attribute data. For example, the inversion unit 124 assigns a hierarchical level number (the top hierarchical level is 0, the value is incremented by 1 for each lower hierarchical level, and the lowest hierarchical level is the maximum level) to each hierarchical level of the attribute data in the reverse order of the generation. In other words, by assigning this hierarchical level number to the data of each hierarchical level, the hierarchical levels are generated in the order from the lowest hierarchical level to the highest hierarchical level.
[0131] The inversion unit 124 supplies the attribute data with the hierarchical hierarchy inverted to the weighting unit 125 .
[0132] The weighting unit 125 performs processing related to weighting. For example, the weighting unit 125 acquires attribute data supplied from the inversion unit 124. The weighting unit 125 derives a weight value for the acquired attribute data. Note that the method of deriving this weight value is arbitrary.
[0133] For example, the weighting unit 125 may derive weight values using reference relationships between sub-hierarchies (Sub LoDs), as in "Method 1-2" in the fifth row from the top of the table shown in Figure 3 described above in <1. Generation of Sub LoD>.
[0134] Furthermore, when attribute data is hierarchically organized (LoD) using Lifting described in Non-Patent Document 2, the weighting unit 125 may reflect the reference relationship of attribute data between sub-hierarchies in the weight value set for each attribute data in the hierarchical organization. In other words, the weighting unit 125 may derive the weight value according to both the reference relationship of attribute data between hierarchies and the reference relationship of attribute data between sub-hierarchies.
[0135] In addition, when attribute data is layered (LoD) using a method compatible with scalable decoding described in Non-Patent Document 3, the weighting unit 125 may reflect the reference relationship of attribute data between sub-layers in the weight value for each layer generated in the layering, as in "Method 1-2-1" in the sixth row from the top of the table shown in FIG. 3.
[0136] Of course, methods other than these may be used. Note that weighting section 125 may omit this weighting.
[0137] The weighting unit 125 predicts the attribute data of each point using the weight values derived as described above and the reference relationship between the attribute data, and derives a predicted value. The weighting unit 125 also derives a difference value between the attribute data of each point and its predicted value.
[0138] The weighting unit 125 supplies the derived attribute data (difference value) to the quantization unit 112 (FIG. 9). The weighting unit 125 may also supply the derived weight value as control information to the quantization unit 112, which may then transmit the control information to the decoding side.
[0139] These processing units (control unit 121 to weighting unit 125) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0140] <Encoding process flow> Next, a description will be given of the processing executed by the encoding device 100. The encoding device 100 encodes point cloud data by executing an encoding process. An example of the flow of the encoding process will be described with reference to the flowchart in FIG.
[0141] When the encoding process starts, in step S101, the position information encoding unit 101 of the encoding device 100 encodes the geometry data (position information) of the input point cloud, and generates encoded data of the geometry data.
[0142] In step S102, the position information decoding unit 102 decodes the encoded data of the geometry data generated in step S101 to generate position information.
[0143] In step S103, the point cloud generation unit 103 performs recolor processing using the attribute data (attribute information) of the input point cloud and the geometry data (decoding result) generated in step S102, and makes the attribute data correspond to the geometry data.
[0144] In step S104, the attribute information encoding unit 104 executes an attribute information encoding process to encode the attribute data recolored in step S103, and generate encoded data of the attribute data. At this time, the attribute information encoding unit 104 performs processing by applying the present technology described above in <1. Generation of Sub LoD>. Details of the attribute information encoding process will be described later.
[0145] In step S105, the bit stream generating unit 105 generates and outputs a bit stream including the coded data of the geometry data generated in step S101 and the coded data of the attribute data generated in step S104.
[0146] When the process of step S105 ends, the encoding process ends.
[0147] By performing the processing of each step in this manner, the encoding device 100 can generate a reference relationship of attribute data within a hierarchy so as to suppress bias in the prediction direction, and can suppress a decrease in encoding efficiency.
[0148] <Attribute information encoding process flow> Next, an example of the flow of the attribute information encoding process executed in step S104 in FIG. 11 will be described with reference to the flowchart in FIG.
[0149] When the attribute information encoding process is started, in step S111, the hierarchical processing unit 111 of the attribute information encoding unit 104 performs hierarchical processing to hierarchically organize the attribute data and derive a difference value of the attribute data of each point. At this time, the hierarchical processing unit 111 performs hierarchical organization by applying the present technology described above in <1. Generation of Sub LoD>. Details of the hierarchical organization process will be described later.
[0150] In step S112, the quantization unit 112 quantizes each of the difference values derived in step S111.
[0151] In step S113, the encoding unit 113 encodes the difference value quantized in step S112 to generate encoded data of the attribute data. If control information is generated in the layering process in step S111, the encoding unit 113 generates encoded data of the attribute data including the control information.
[0152] When the process of step S113 ends, the attribute information encoding process ends, and the process returns to FIG.
[0153] By processing each step in this manner, the attribute information encoding unit 104 can generate a reference relationship of attribute data within a hierarchy so as to suppress bias in the prediction direction, thereby suppressing a decrease in encoding efficiency.
[0154] <Flow of layering process> Next, an example of the flow of the layering process executed in step S111 of FIG. 12 will be described with reference to the flowchart of FIG.
[0155] When the hierarchical processing is started, in step S121, the control unit 121 of the hierarchical processing unit 111 processes the attribute data of all points, and performs the processes of steps S122 to S126 to generate the first layer (LoD).
[0156] This first layer (the layer generated first) becomes the lowest layer in the hierarchical attribute data by the process in step S127 described later. In other words, it can be said that the control unit 121 sets the processing target LoD to the lowest layer.
[0157] In step S122, the hierarchical processing unit 122 sets a reference point from among the points to be processed. A reference point is a point whose attribute data is referenced when predicting the attribute data of a prediction point. A prediction point is a point at which prediction of attribute data is performed in this layer. In other words, it can be said that the hierarchical processing unit 122 sets each point to be processed as either a prediction point or a reference point.
[0158] In step S123, the control unit 121 determines whether or not to generate a sub-layer (Sub LoD) in this layer. The control unit 121 determines whether or not to create a sub-layer in this layer based on any information, such as settings by a user or an application. If it is determined that a sub-layer is to be created, the process proceeds to step S124.
[0159] In step S124, the Sub LoD generator 123 generates a sub-hierarchy in the hierarchy to be processed by using the geometry data. That is, the Sub LoD generator 123 generates a reference relationship of attribute data in the hierarchy (LoD) as in "Method 1" in the top row of the table shown in Fig. 3 described above in <1. Generation of Sub LoD>.
[0160] This generation method is arbitrary. For example, the Sub LoD generation unit 123 may apply "Method 1-1" in the second row from the top of the table shown in Fig. 3, or "Method 1-1-1" in the third row from the top of the table, or "Method 1-1-2" in the fourth row from the top of the table. Of course, the Sub LoD generation unit 123 may apply other methods.
[0161] Once the sub-hierarchy is generated, processing proceeds to step S125.
[0162] Also, if it is determined in step S123 that sub-layering is not to be performed on the hierarchical layer being processed, the process of step S124 is skipped and the process proceeds to step S125.
[0163] In step S125, the control unit 121 processes the attribute data of the reference point selected in step S122, and controls so that each process from step S122 to step S126 is performed to generate the next hierarchy (LoD).
[0164] The next layer newly selected as the processing target becomes the layer one level higher than the layer previously selected as the processing target in the hierarchical attribute data by the processing in step S127 described later. In other words, it can be said that the control unit 121 updates the processing target LoD to the next higher layer.
[0165] In step S126, the control unit 121 determines whether all points have been processed. By repeating the above-mentioned process, all points are set as prediction points (there may be cases where prediction is not performed for the last point). In other words, the control unit 121 determines whether all hierarchies have been generated. If it is determined that there are points that have not yet been selected as prediction points and hierarchization is not complete, the process returns to step S122.
[0166] That is, the processes of steps S122 to S126 are executed for the next hierarchical layer that is newly selected as the processing target in step S125. That is, the point set as the previous reference point is set as either a prediction point or a reference point. In this way, the processes of steps S122 to S126 are recursively repeated for the point set as the reference point, each hierarchical layer and each sub-hierarchical layer are generated, and reference relationships between hierarchical layers and between sub-hierarchical layers are generated.
[0167] By repeatedly executing the processes of steps S122 to S126, all hierarchical levels (and all sub-hierarchical levels) are generated, and if it is determined in step S126 that all points have been processed, the process proceeds to step S127.
[0168] In step S127, the inversion unit 124 inverts the hierarchy of the generated attribute data, and assigns hierarchy numbers to each hierarchy in the reverse order to the generation order.
[0169] In step S128, the weighting unit 125 derives a weighting value for the attribute data of each point. At that time, the weighting unit 125 derives a weighting value that reflects the reference relationship between the sub-hierarchies.
[0170] Furthermore, the weighting unit 125 predicts attribute data for each point using the derived weight value and the reference relationship between the attribute data, and derives a difference value between the attribute data and the predicted value.
[0171] In step S129, the control unit 121 generates control information regarding the sub-hierarchical division of the attribute data, and transmits it to the decoding side.
[0172] When the process of step S129 ends, the layering process ends, and the process returns to FIG.
[0173] By performing the processes in each step in this manner, the layering processing unit 111 can generate a reference relationship of attribute data within a layer so as to suppress bias in the prediction direction. Therefore, the encoding device 100 can suppress a decrease in encoding efficiency.
[0174] <3. Second embodiment> <Decryption device> Next, another example of a device to which the present technology described above in <1. Generation of Sub LoD> is applied will be described. Fig. 14 is a block diagram showing an example of a configuration of a decoding device, which is one aspect of an information processing device to which the present technology is applied. The decoding device 200 shown in Fig. 14 is a device that decodes encoded data of a point cloud (3D data). The decoding device 200 decodes encoded data of a point cloud by applying the present technology described above in <1. Generation of Sub LoD>.
[0175] Note that Fig. 14 shows the main processing units, data flows, and the like, and is not limited to all that is shown in Fig. 14. That is, in the decoding device 200, there may be processing units that are not shown as blocks in Fig. 14, and there may be processing and data flows that are not shown as arrows or the like in Fig. 14.
[0176] As shown in FIG. 14, the decoding device 200 includes a decoding target LoD depth setting unit 201 , an encoded data extraction unit 202 , a position information decoding unit 203 , an attribute information decoding unit 204 , and a point cloud generation unit 205 .
[0177] The decoding target LoD depth setting unit 201 performs processing related to setting the depth of the layer (LoD) to be decoded. For example, the decoding target LoD depth setting unit 201 sets up to which layer the coded data of the point cloud held in the coded data extraction unit 202 is to be decoded. The method of setting the layer depth to be decoded is arbitrary.
[0178] For example, the decoding target LoD depth setting unit 201 may set the layer depth based on an instruction related to the layer depth from an external source such as a user or an application. Also, the decoding target LoD depth setting unit 201 may obtain and set the layer depth to be decoded based on any information such as an output image.
[0179] For example, the decoding target LoD depth setting unit 201 may set the hierarchical depth to be decoded based on the viewpoint position, direction, angle of view, viewpoint movement (movement, pan, tilt, zoom), etc. of the two-dimensional image generated from the point cloud.
[0180] The data unit for setting the hierarchical depth to be decoded is arbitrary. For example, the decoding target LoD depth setting unit 201 can set the hierarchical depth for the entire point cloud, can set the hierarchical depth for each object, or can set the hierarchical depth for each partial area in the object. Of course, the hierarchical depth can be set in a data unit other than these examples.
[0181] The coded data extraction unit 202 acquires and holds a bitstream input to the decoding device 200. The coded data extraction unit 202 extracts coded data of geometry data (position information) and attribute data (attribute information) from the top to the layer specified by the decoding target LoD depth setting unit 201 from the bitstream held by it. The coded data extraction unit 202 supplies the coded data of the extracted geometry data to the position information decoding unit 203. The coded data extraction unit 202 supplies the coded data of the extracted attribute data to the attribute information decoding unit 204.
[0182] The position information decoding unit 203 acquires the coded data of the geometry data supplied from the coded data extraction unit 202. The position information decoding unit 203 decodes the coded data of the geometry data to generate geometry data (decoded result). This decoding method may be any method similar to that used by the position information decoding unit 102 of the encoding device 100. The position information decoding unit 203 supplies the generated geometry data (decoded result) to the attribute information decoding unit 204 and the point cloud generation unit 205.
[0183] The attribute information decoding unit 204 acquires the coded data of the attribute data supplied from the coded data extraction unit 202. The attribute information decoding unit 204 acquires the geometry data (decoded result) supplied from the position information decoding unit 203. The attribute information decoding unit 204 uses the position information (decoded result) to decode the coded data of the attribute data by the method to which the present technology is applied described above in <1. Generation of Sub LoD>, and generates attribute data (decoded result). The attribute information decoding unit 204 supplies the generated attribute data (decoded result) to the point cloud generation unit 205.
[0184] The point cloud generating unit 205 acquires geometry data (decoding result) supplied from the position information decoding unit 203. The point cloud generating unit 205 acquires attribute data (decoding result) supplied from the attribute information decoding unit 204. The point cloud generating unit 205 generates a point cloud (decoding result) using the geometry data (decoding result) and the attribute data (decoding result). The point cloud generating unit 205 outputs the generated point cloud (decoding result) data to the outside of the decoding device 200.
[0185] With this configuration, the decoding device 200 can perform inverse hierarchical construction by applying a plurality of hierarchical construction methods, and can therefore correctly inversely hierarchical construction of attribute data hierarchically constructed by applying a plurality of hierarchical construction methods. In other words, it is possible to correctly decode encoded data that has been encoded by applying such a plurality of hierarchical construction methods. This makes it possible to suppress a decrease in encoding efficiency.
[0186] In addition, these processing units (the decoding target LoD depth setting unit 201 to the point cloud generating unit 205) have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0187] <Attribute information decoding part> Fig. 15 is a block diagram showing a main configuration example of the attribute information decoding unit 204 (Fig. 14). Note that Fig. 15 shows main processing units, data flows, etc., and is not limited to all that are shown in Fig. 15. In other words, in the attribute information decoding unit 204, there may be processing units not shown as blocks in Fig. 15, and there may be processing and data flows not shown as arrows, etc. in Fig. 15.
[0188] As shown in FIG. 15, the attribute information decoding unit 204 includes a decoding unit 211, an inverse quantization unit 212, and an inverse layer processing unit 213.
[0189] The decoding unit 211 performs processing related to decoding of the coded data of the attribute data. For example, the decoding unit 211 obtains the coded data of the attribute data supplied to the attribute information decoding unit 204.
[0190] The decoding unit 211 decodes the coded data of the attribute data to generate attribute data (decoded result). Any decoding method may be used as long as it corresponds to the coding method used by the coding unit 113 (FIG. 9) of the coding device 100. The generated attribute data (decoded result) corresponds to the attribute data before coding, and is a difference value between the attribute data and its predicted value, and is quantized, as described in the first embodiment. The decoding unit 211 supplies the generated attribute data (decoded result) to the inverse quantization unit 212.
[0191] If the encoded data of the attribute data includes control information related to weight values and control information related to hierarchical organization of the attribute data, the decoding unit 211 also supplies the control information to the inverse quantization unit 212 .
[0192] The inverse quantization unit 212 performs processing related to inverse quantization of attribute data. For example, the inverse quantization unit 212 acquires attribute data (decoding result) supplied from the decoding unit 211. When control information is supplied from the decoding unit 211, the inverse quantization unit 212 also acquires the control information.
[0193] The inverse quantization unit 212 inverse quantizes the attribute data (decoded result). The inverse quantization unit 212 performs this inverse quantization using a method corresponding to the quantization by the quantization unit 112 (FIG. 9) of the encoding device 100. The inverse quantization unit 212 supplies the inverse quantized attribute data to the inverse hierarchical processing unit 213. Furthermore, when control information is acquired from the decoding unit 211, the inverse quantization unit 212 also supplies the control information to the inverse hierarchical processing unit 213.
[0194] The inverse hierarchical processing unit 213 obtains the inversely quantized attribute data supplied from the inverse quantization unit 212. As described above, this attribute data is a difference value. The inverse hierarchical processing unit 213 also obtains geometry data (decoded result) supplied from the position information decoding unit 203. The inverse hierarchical processing unit 213 performs inverse hierarchical processing on the obtained attribute data (difference value) using the geometry data. This inverse hierarchical processing is the inverse process of the hierarchical processing performed by the hierarchical processing unit 111 (FIG. 9) of the encoding device 100.
[0195] At this time, the inverse layer processing unit 213 performs inverse layering by applying the present technology described above in <1. Generation of Sub LoD>. For example, the inverse layer processing unit 213 generates sub-layers (Sub LoD) within the layer of attribute data, generates reference relationships of attribute data between the sub-layers, predicts attribute data using the reference relationships, and performs inverse layering. The inverse layer processing unit 213 supplies the inverse layered attribute data to the point cloud generating unit 205 (FIG. 14) as a decoding result.
[0196] By performing the inverse hierarchical structure as described above, the attribute information decoding unit 204 can correctly inversely hierarchical structure the attribute data that has been hierarchically structured or sub-hierarchically structured by applying the above-mentioned "Method 1" etc. Therefore, the attribute information decoding unit 204 can suppress a decrease in the coding efficiency.
[0197] In addition, these processing units (the decoding unit 211 to the reverse layering processing unit 213) may have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.
[0198] <De-hierarchical layer processing unit> Fig. 16 is a block diagram showing an example of the main configuration of the inverse hierarchical layer processing unit 213 (Fig. 15). Note that Fig. 16 shows the main processing units, data flows, etc., and is not limited to all that is shown in Fig. 16. In other words, in the inverse hierarchical layer processing unit 213, there may be processing units that are not shown as blocks in Fig. 16, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 16.
[0199] As shown in FIG. 16, the inverse layer processing unit 213 includes a control unit 221, a layer processing unit 222, a Sub LoD generating unit 223, an inverter 224, and an inverse layer processing unit 225.
[0200] The control unit 221 performs processing related to the control of hierarchical layering. For example, the control unit 221 acquires inversely quantized attribute data supplied from the inverse quantization unit 212 (FIG. 15). The control unit 221 also acquires geometry data (decoding result) supplied from the position information decoding unit 203. The control unit 221 supplies the acquired attribute data and geometry data to the hierarchical layering processing unit 222. Furthermore, when control information is supplied from the inverse quantization unit 212, the control unit 221 also acquires the control information and supplies it to the hierarchical layering processing unit 222.
[0201] Furthermore, the control unit 221 controls the layering processing unit 222 and the Sub LoD generating unit 223 to perform layering and sub-layering on the attribute data, and to generate layers and sub-layers having the same structure as the layers and sub-layers generated during encoding. For example, the control unit 221 applies the present technology described above in <1. Generation of Sub LoD> to perform layering and sub-layering. For example, the control unit 221 causes sub-layering to be performed on a desired layer. That is, the control unit 221 can cause sub-layering to be performed on some layers, or sub-layering to be performed on all layers.
[0202] For example, the control unit 221 may perform sub-hierarchical division based on control information related to the sub-hierarchical division of attribute data, as in "Method 2-2" in the tenth row from the top of the table shown in FIG.
[0203] For example, the control unit 221 may cause the attribute data to be sub-hierarchized based on control information (eg, sub_lod_enable_flag) indicating whether or not sub-hierarchization of the attribute data is permitted.
[0204] Furthermore, the control unit 221 may perform sub-hierarchical division (sampling) of the attribute data based on control information (for example, sub_lod_distance) indicating the sampling interval of the attribute data.
[0205] Furthermore, the control unit 221 may perform sub-hierarchical division (sampling) of the attribute data based on control information (for example, sub_lod_mode) indicating a method of sampling the attribute data.
[0206] By using such control information transmitted from the encoding side, the control unit 221 can more easily perform sub-hierarchical division similar to that on the encoding side.
[0207] The layering processing unit 222 performs processing related to layering of attribute data. For example, the layering processing unit 222 obtains attribute data and geometry data (decoding results) supplied from the control unit 221.
[0208] The hierarchical processing unit 222 hierarchizes the acquired attribute data using the acquired geometry data under the control of the control unit 221. The hierarchical method may be the same as the hierarchical processing by the hierarchical processing unit 122. That is, the hierarchical processing unit 222 hierarchizes the attribute data into a structure similar to that generated by the hierarchical processing unit 122. In other words, the hierarchical structure of the attribute data generated by the hierarchical processing unit 122 is reproduced. The hierarchical processing unit 222 supplies the hierarchical attribute data and geometry data to the Sub LoD generation unit 223.
[0209] The Sub LoD generating unit 223 performs processing related to sub-hierarchical generation. For example, the Sub LoD generating unit 223 obtains hierarchical attribute data and geometry data supplied from the hierarchical generation processing unit 222.
[0210] The Sub LoD generating unit 223 generates a sub-hierarchy (Sub LoD) within the hierarchy of the attribute data in accordance with the control of the control unit 221. The Sub LoD generating unit 223 performs such sub-hierarchy generation by using geometry data.
[0211] In other words, the Sub LoD generation unit 223 generates a sub-hierarchy (Sub LoD) of attribute data within a hierarchy (LoD) and generates reference relationships between the sub-hierarchies, as in "Method 2-1" in the ninth row from the top of the table shown in Figure 3 described above in <1. Generation of Sub LoD>.
[0212] In this way, the Sub LoD generating unit 223 can generate a reference relationship of attribute data within a hierarchy so as to suppress bias in the prediction direction, and can suppress a decrease in coding efficiency.
[0213] The Sub LoD generation unit 223 performs this sub-hierarchical generation using the same method as the sub-hierarchical generation by the Sub LoD generation unit 123. That is, the Sub LoD generation unit 223 generates a sub-hierarchical structure similar to that generated by the Sub LoD generation unit 123. In other words, the sub-hierarchical structure of the attribute data generated by the Sub LoD generation unit 123 is reproduced.
[0214] For example, the Sub LoD generation unit 223 may recursively repeat the process of sampling (selecting) a portion of the attribute data for each point in a hierarchy to create a sub-hierarchy, as in "Method 1-1" in the second row from the top of the table shown in Figure 3.
[0215] In addition, the Sub LoD generation unit 223 may generate a sub-hierarchy by arranging the attribute data of the hierarchy to be processed in Morton order and sampling a portion of it at equal intervals, as in "Method 1-1-1" in the third row from the top of the table shown in Figure 3.
[0216] In addition, the Sub LoD generation unit 223 may generate a sub-hierarchy by arranging the attribute data of the hierarchy to be processed in Morton order and sampling some of it at non-equidistant intervals, as in "Method 1-1-2" in the fourth row from the top of the table shown in Figure 3.
[0217] Of course, methods other than these may be used. If sub-layering is not permitted by the control unit 221, the Sub LoD generation unit 223 can omit sub-layering. In other words, the Sub LoD generation unit 223 can perform sub-layering only on layers permitted by the control unit 221.
[0218] The Sub LoD generating unit 223 supplies the attribute data thus sub-hierarchized under the control of the control unit 221 to the inverting unit 224 .
[0219] The inversion unit 224 performs processing related to hierarchical inversion. For example, the inversion unit 224 obtains attribute data supplied from the Sub LoD generation unit 223.
[0220] The inversion unit 224 inverts the hierarchy of the attribute data in the same manner as the inversion unit 124. For example, the inversion unit 224 assigns a hierarchy number (a number for identifying a hierarchy in which the topmost hierarchy is 0, the value is incremented by 1 for each hierarchy level, and the lowest hierarchy is the maximum value) to each hierarchy of the attribute data in the reverse order of the generation order.
[0221] The inversion unit 224 supplies the attribute data with the inverted hierarchy to the inverse hierarchy processing unit 225 .
[0222] The inverse hierarchical processing unit 225 performs processing related to inverse hierarchical processing. For example, the inverse hierarchical processing unit 225 acquires attribute data supplied from the inverter 224. This attribute data is composed of difference values between the attribute data and its predicted values, and has a hierarchical structure generated (reproduced) by the hierarchical processing unit 222 and a sub-hierarchical structure generated (reproduced) by the Sub LoD generating unit 223. The inverse hierarchical processing unit 225 inversely processes the acquired attribute data, and generates (restores) the attribute data of each point.
[0223] That is, the inverse hierarchical processing unit 225 predicts attribute data for each point, and derives (restores) the attribute data for each point from the difference value using the derived predicted value. At that time, the inverse hierarchical processing unit 225 predicts attribute data (derives predicted values of attribute data) using the reference relationships between hierarchies and sub-hierarchies reproduced as described above, as in "Method 2" in the eighth row from the top of the table shown in Fig. 3.
[0224] For example, the inverse layering processing unit 225 derives predicted values of attribute data for each point for each layer, from the highest layer to the lowest layer (within a layer, for each sub-layer, from the highest sub-layer to the lowest sub-layer).
[0225] The inverse layer processing unit 225 then adds the derived predicted value to the difference value corresponding to that point to derive the attribute data for that point. In this manner, the inverse layer processing unit 225 generates (restores) the attribute data for each point.
[0226] The inverse layer processing unit 225 supplies the attribute data (decoded result) generated (restored) by the inverse layer processing as described above to the point cloud generating unit 205 (FIG. 14).
[0227] By performing the inverse hierarchical structure as described above, the inverse hierarchical structure processor 225 can correctly inversely hierarchical structure the attribute data that has been hierarchically structured or sub-hierarchically structured by applying the above-mentioned "Method 1" or the like, and therefore the decoding device 200 can suppress a decrease in encoding efficiency.
[0228] These processing units (the control unit 221 to the reverse hierarchical processing unit 225) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and by executing a program.
[0229] <Decryption process flow> Next, a description will be given of the processing executed by the decoding device 200. The decoding device 200 decodes the encoded data of the point cloud by executing a decoding process. An example of the flow of the decoding process will be described with reference to the flowchart in FIG.
[0230] When the decoding process is started, the decoding target LoD depth setting unit 201 of the decoding device 200 sets the LoD depth to be decoded (that is, the range of layers to be decoded) in step S201.
[0231] In step S202, the coded data extraction unit 202 acquires and holds the bitstream, and extracts coded data of geometry data (position information) and attribute data (attribute information) from the top layer to the layer of the LoD depth set in step S201. The layer of the extracted geometry data and the layer (number of layers) of the attribute data may or may not match each other.
[0232] In step S203, the position information decoding unit 203 decodes the encoded data of the geometry data extracted in step S202, and generates geometry data (decoded result).
[0233] In step S204, the attribute information decoding unit 204 decodes the coded data of the attribute data extracted in step S202 to generate attribute data (decoded result). At this time, the attribute information decoding unit 204 performs processing by applying the present technology described above in <1. Generation of Sub LoD>. Details of the attribute information decoding process will be described later.
[0234] In step S205, the point cloud generating unit 205 generates and outputs point cloud data (decoding result) using the geometry data (decoding result) generated in step S203 and the attribute data (decoding result) generated in step S204.
[0235] When the process of step S205 ends, the decoding process ends.
[0236] By performing the processes at each step in this manner, the decoding device 200 can correctly de-hierarchize the attribute data that has been hierarchized / sub-hierarchized by applying the above-mentioned "Method 1" etc. Therefore, the decoding device 200 can suppress a decrease in encoding efficiency.
[0237] <Flow of attribute information decryption process> Next, an example of the flow of the attribute information decoding process executed in step S204 in FIG. 17 will be described with reference to the flowchart in FIG.
[0238] When the attribute information decoding process is started, in step S211, the decoding unit 211 of the attribute information decoding unit 204 decodes the coded data of the attribute data to generate attribute data (decoded result). This attribute data (decoded result) is quantized as described above.
[0239] In step S212, the inverse quantization unit 212 executes an inverse quantization process to inverse quantize the attribute data (decoding result) generated in step S211. This inverse quantized attribute data is a difference value.
[0240] In step S213, the inverse hierarchical processing unit 213 performs an inverse hierarchical processing to inversely hierarchically process the attribute data inversely quantized in step S212, and derive the attribute data of each point. At this time, the inverse hierarchical processing unit 213 performs the inverse hierarchical processing by applying the present technology described above in <1. Generation of Sub LoD>. The details of the inverse hierarchical processing will be described later.
[0241] When the process of step S213 ends, the attribute information decoding process ends, and the process returns to FIG.
[0242] By performing the processes at each step in this manner, the attribute information decoding unit 204 can correctly de-hierarchize the attribute data that has been hierarchized / sub-hierarchized by applying the above-mentioned "Method 1" etc. Therefore, the decoding device 200 can suppress a decrease in the encoding efficiency.
[0243] <Flow of reverse hierarchy processing> Next, an example of the flow of the hierarchical inverse process executed in step S213 in FIG. 18 will be described with reference to the flowchart in FIG.
[0244] When the inverse layering process is started, in step S221, the control unit 221 of the inverse layering processing unit 213 processes the attribute data (difference values) of all points, as in step S121 (FIG. 13), and performs the processes of steps S222 to S226 to generate the first layer (LoD).
[0245] This first layer (the layer generated first) becomes the lowest layer in the hierarchical attribute data by the process in step S227 described later. In other words, it can be said that the control unit 221 sets the processing target LoD to the lowest layer.
[0246] In step S222, the hierarchical processing unit 222 sets reference points from among the points to be processed, similar to the case of step S122 (FIG. 13). In other words, it can be said that the hierarchical processing unit 222 sets each point to be processed as either a prediction point or a reference point.
[0247] In step S223, the control unit 221 determines whether or not to generate a sub-layer (Sub LoD) in this layer, similarly to the case of step S123 (FIG. 13). The control unit 221 can make this determination based on any information, etc. For example, the control unit 221 determines whether or not to create a sub-layer in this layer based on control information, etc. transmitted from the encoding side. If it is determined that a sub-layer is to be created, the process proceeds to step S224.
[0248] In step S224, the Sub LoD generator 223 generates a sub-hierarchy in the hierarchy to be processed by using the geometry data, similar to the case of step S124 (FIG. 13). That is, the Sub LoD generator 223 generates a sub-hierarchy (Sub LoD) of attribute data in the hierarchy (LoD) and generates reference relationships between the sub-hierarchies, as in "Method 2-1" in the ninth row from the top of the table shown in FIG. 3 described above in <1. Generation of Sub LoD>.
[0249] That is, in step S224, the Sub LoD generation unit 223 performs this sub-hierarchical generation using the same method as the sub-hierarchical generation performed in step S124. That is, the Sub LoD generation unit 223 generates a sub-hierarchical structure similar to that generated in step S124. In other words, the sub-hierarchical structure of the attribute data generated in step S124 is reproduced.
[0250] For example, the Sub LoD generation unit 223 may apply "Method 1-1" in the second row from the top of the table shown in Fig. 3, or may apply "Method 1-1-1" in the third row from the top of the table, or may apply "Method 1-1-2" in the fourth row from the top of the table. Of course, the Sub LoD generation unit 223 may apply other methods.
[0251] Once the sub-hierarchy is generated, processing proceeds to step S225.
[0252] Also, if it is determined in step S223 that sub-layering is not to be performed on the layer being processed, the process of step S224 is skipped and the process proceeds to step S225.
[0253] In step S225, similarly to step S125 (FIG. 13), the control unit 221 processes the attribute data of the reference point selected in step S222, and causes the processes of steps S222 to S226 to be performed in order to generate the next hierarchy (LoD).
[0254] The next layer newly selected as the processing target becomes the layer one level higher than the layer previously selected as the processing target in the hierarchical attribute data by the processing in step S227 described later. In other words, it can be said that the control unit 221 updates the processing target LoD to the next higher layer.
[0255] In step S226, the control unit 221 determines whether all points have been processed, similarly to step S126 (FIG. 13). By repeating the above-mentioned process, all points are set as prediction points (there may be cases where prediction is not performed for the last point). That is, the control unit 221 determines whether all hierarchies have been generated. If it is determined that there are points that have not yet been selected as prediction points and hierarchization is not complete, the process returns to step S222.
[0256] That is, the processes of steps S222 to S226 are executed for the next hierarchical layer that is newly selected as the processing target in step S225. That is, the point set as the previous reference point is set as either a prediction point or a reference point. In this manner, the processes of steps S222 to S226 are recursively repeated for the point set as the reference point, each hierarchical layer and each sub-hierarchical layer are generated, and reference relationships between hierarchical layers and between sub-hierarchical layers are generated (reproduced).
[0257] By repeatedly executing the processes of steps S222 through S226, all hierarchical levels (and all sub-hierarchical levels) are generated (reproduced), and if it is determined in step S226 that all points have been processed, the process proceeds to step S227.
[0258] In step S227, the inversion unit 224 inverts the hierarchy of the generated attribute data, as in step S127 (FIG. 13), and assigns hierarchy numbers to each hierarchy in the reverse order of generation. As a result, attribute data is generated (reproduced), which is composed of difference values between attribute data and their predicted values, and has the hierarchical structure generated (reproduced) in step S222 and the sub-hierarchical structure generated (reproduced) in step S224.
[0259] In step S228, the inverse hierarchical processing unit 225 inversely hierarchically processes this generated attribute data, and generates (restores) attribute data for each point.
[0260] That is, the inverse hierarchical processing unit 225 predicts attribute data for each point, and derives (restores) the attribute data for each point from the difference value using the derived predicted value. At that time, the inverse hierarchical processing unit 225 predicts attribute data (derives predicted values of attribute data) using the reference relationships between hierarchies and sub-hierarchies reproduced as described above, as in "Method 2" in the eighth row from the top of the table shown in Fig. 3.
[0261] For example, the inverse layering processing unit 225 derives predicted values of attribute data for each point for each layer, from the highest layer to the lowest layer (within a layer, for each sub-layer, from the highest sub-layer to the lowest sub-layer).
[0262] The inverse layer processing unit 225 then adds the derived predicted value to the difference value corresponding to that point to derive the attribute data for that point. In this manner, the inverse layer processing unit 225 generates (restores) the attribute data for each point.
[0263] When attribute data for all points has been generated (restored) and the inverse hierarchical structure processing is completed, the inverse hierarchical structure processing ends and the processing returns to FIG.
[0264] By performing the processing of each step in this manner, the inverse layering processor 213 can correctly inversely layer the attribute data that has been layered or sub-layered by applying the above-mentioned "Method 1" etc. Therefore, the decoding device 200 can suppress a decrease in encoding efficiency.
[0265] <4. Notes> <Hierarchization / reverse hierarchy method> In the above, Lifting has been used as an example of a method for creating a hierarchy and de-creasing a hierarchy for attribute information, but this technology can be applied to any technology for creating a hierarchy for attribute information. In other words, the method for creating a hierarchy and de-creasing a hierarchy for attribute information may be other than Lifting.
[0266] <Computer> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, etc., capable of executing various functions by installing various programs.
[0267] FIG. 20 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.
[0268] In a computer 900 shown in FIG. 20, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.
[0269] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.
[0270] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0271] In the computer configured as above, the series of processes described above are performed by the CPU 901, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data and the like necessary for the CPU 901 to execute various processes as appropriate.
[0272] The program executed by the computer can be applied by recording it on a removable medium 921 such as a package medium, for example. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by mounting the removable medium 921 on the drive 915.
[0273] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.
[0274] In addition, this program can also be installed in advance in the ROM 902 or the storage unit 913.
[0275] <Target of application of this technology> Although the above describes the application of this technology to the encoding and decoding of point cloud data, this technology is not limited to these examples and can be applied to the encoding and decoding of 3D data of any standard. In other words, as long as it does not conflict with the above-mentioned technology, various processes such as encoding and decoding methods and specifications of various data such as 3D data and metadata are arbitrary. In addition, as long as it does not conflict with the present technology, some of the above-mentioned processes and specifications may be omitted.
[0276] Furthermore, although the encoding device 100 and the decoding device 200 have been described above as application examples of the present technology, the present technology can be applied to any configuration.
[0277] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) in satellite broadcasting, wired broadcasting such as cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media.
[0278] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).
[0279] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and cooperate with each other to perform processing via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (video images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.
[0280] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.
[0281] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications are also arbitrary.
[0282] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Thus, the value that this "flag" can take may be, for example, a binary value of 1 / 0, or may be three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, it is assumed that identification information (including flags) is not only included in the bit stream, but also includes difference information of the identification information with respect to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information with respect to the reference information.
[0283] Also, various information (metadata, etc.) related to the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, data associated with each other may be combined into one piece of data, or each piece of data may be made into separate data. For example, information associated with coded data (image) may be transmitted on a transmission path different from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium different from that of the coded data (image) (or on a different recording area of the same recording medium). Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in any unit, such as multiple frames, one frame, or a part of a frame.
[0284] In this specification, the terms "combine," "multiplex," "append," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.
[0285] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.
[0286] For example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).
[0287] Also, for example, the above-mentioned program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.
[0288] Also, for example, each step of one flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.
[0289] Also, for example, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel, or individually at a required timing such as when a call is made. In other words, the processing of each step may be executed in an order different from the order described above, so long as no contradiction occurs. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.
[0290] Also, for example, a plurality of techniques related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any of a plurality of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in other embodiments. Also, a part or all of any of the above-mentioned present technologies can be implemented in combination with other technologies not described above.
[0291] The present technology can also be configured as follows. (1) a hierarchical generation unit that hierarchically generates attribute information of a point cloud that represents a three-dimensional object as a set of points, and generates a reference relationship between the attribute information between layers; a sub-hierarchy generation unit that generates sub-hierarchies of the attribute information in the hierarchy of the attribute information generated by the hierarchy generation unit and generates a reference relationship of the attribute information between the sub-hierarchies; An information processing device comprising: (2) The sub-hierarchy generating unit generates sub-hierarchies of the attribute information by arranging the attribute information of the processing target hierarchy in Morton's order and sampling the attribute information at equal intervals. An information processing device as described in (1). (3) The sub-hierarchy generating unit generates sub-hierarchies of the attribute information by arranging the attribute information of the processing target hierarchy in Morton's order and sampling the attribute information at non-equidistant intervals. An information processing device as described in (1). (4) a weighting unit that derives a weighting value for each of the attribute information by updating a weighting value for each of the layers in accordance with a reference relationship of the attribute information between the sub-layers generated by the sub-layer generating unit; The information processing device according to any one of (1) to (3), further comprising: (5) a weighting unit that derives a weight value for each of the attribute information in accordance with a reference relationship of the attribute information between the hierarchies generated by the hierarchical generation unit and a reference relationship of the attribute information between the sub-hierarchies generated by the sub-hierarchical generation unit. The information processing device according to any one of (1) to (3), further comprising: (6) an encoding unit that encodes the attribute information and generates encoded data of the attribute information; The information processing device according to any one of (1) to (5), further comprising: (7) A generating unit that generates control information regarding sub-hierarchical division of the attribute information, The encoding unit generates the encoded data including the control information generated by the generation unit. (6) An information processing device. (8) The generating unit generates control information indicating whether or not sub-hierarchical layering of the attribute information is permitted. (7) An information processing device. (9) The generating unit generates control information indicating an interval for sampling the attribute information by the sub-hierarchical layering unit. An information processing device according to (7) or (8). (10) The generating unit generates control information indicating a method for sampling the attribute information by the sub-hierarchical generation unit. An information processing device according to any one of (7) to (9). (11) hierarchizing attribute information of a point cloud that represents a three-dimensional object as a set of points, and generating a reference relationship between the attribute information between layers; In the generated hierarchy of the attribute information, the attribute information of the hierarchy is divided into sub-hierarchies, and a reference relationship of the attribute information between the sub-hierarchies is generated. Information processing methods.
[0292] (12) a hierarchical generation unit that hierarchically generates attribute information of a point cloud that represents a three-dimensional object as a set of points, and generates a reference relationship between the attribute information between layers; a sub-hierarchy generation unit that generates sub-hierarchies of the attribute information in the hierarchy of the attribute information generated by the hierarchy generation unit and generates reference relationships of the attribute information between the sub-hierarchies; a de-hierarchicalization unit that de-hierarchizes the attribute information based on a reference relationship of the attribute information between the hierarchies generated by the hierarchicalization unit and a reference relationship of the attribute information between the sub-hierarchies generated by the sub-hierarchicalization unit; An information processing device comprising: (13) The sub-hierarchy generating unit generates sub-hierarchies of the attribute information by arranging the attribute information of the processing target hierarchy in Morton order and sampling the attribute information at equal intervals. An information processing device according to (12). (14) The sub-hierarchy generating unit generates sub-hierarchies of the attribute information by arranging the attribute information of the processing target hierarchy in Morton order and sampling the attribute information at non-equidistant intervals. An information processing device according to (12). (15) A decoding unit that decodes the encoded data of the attribute information and restores the attribute information. The information processing device according to any one of (12) to (14), further comprising: (16) The decoding unit decodes the encoded data to restore control information related to sub-hierarchical division of the attribute information, The sub-hierarchical layer generation unit generates a sub-hierarchical layer for the attribute information based on the control information restored by the decoding unit. An information processing device according to (15). (17) The sub-layer generating unit generates a sub-layer for the attribute information based on control information indicating whether sub-layer generation of the attribute information restored by the decoding unit is permitted. An information processing device according to (16). (18) The sub-layer generation unit generates a sub-layer for the attribute information based on control information indicating a sampling interval of the attribute information restored by the decoding unit. The information processing device according to (16) or (17). (19) The sub-hierarchical layer generation unit generates a sub-hierarchical layer for the attribute information based on control information indicating a sampling method for the attribute information restored by the decoding unit. An information processing device according to any one of (16) to (18). (20) hierarchizing attribute information of a point cloud that represents a three-dimensional object as a set of points, and generating a reference relationship between the attribute information between layers; In the generated hierarchy of the attribute information, the attribute information of the hierarchy is divided into sub-hierarchies, and a reference relationship of the attribute information between the sub-hierarchies is generated; The attribute information is inverted into a hierarchy based on the reference relationship of the attribute information between the generated hierarchies and the reference relationship of the attribute information between the generated sub-hierarchies. Information processing methods. [Explanation of symbols]
[0293] 100 encoding device, 101 position information encoding unit, 102 position information decoding unit, 103 point cloud generation unit, 104 attribute information encoding unit, 105 bit stream generation unit, 111 hierarchical processing unit, 112 quantization unit, 113 encoding unit, 121 control unit, 122 hierarchical processing unit, 123 Sub LoD generation unit, 124 inversion unit, 125 weighting unit, 200 decoding device, 201 decoding target LOD depth setting unit, 202 encoded data extraction unit, 203 position information decoding unit, 204 attribute information decoding unit, 205 point cloud generation unit, 211 decoding unit, 212 inverse quantization unit, 213 inverse hierarchical processing unit, 221 control unit, 222 hierarchical processing unit, 223 Sub LoD generation unit, 224 Inversion section, 225 Inverse layer processing section< / lifting> < / octree>
Claims
1. a hierarchical generation unit that hierarchically generates attribute information of a point cloud that represents an object of a three-dimensional shape as a set of points, and generates a reference relationship between the attribute information between layers; a sub-hierarchy generation unit that generates sub-hierarchies of the attribute information in the hierarchy of the attribute information generated by the hierarchy generation unit and generates a reference relationship of the attribute information between the sub-hierarchies; An information processing device comprising:
2. The sub-hierarchy generating unit arranges the attribute information of the processing target hierarchy in Morton order and samples the attribute information at equal intervals to generate a sub-hierarchy for the attribute information. The information processing device according to claim 1 .
3. The sub-hierarchy generating unit arranges the attribute information of the processing target hierarchy in Morton order and samples the attribute information at non-equidistant intervals to generate sub-hierarchies of the attribute information. The information processing device according to claim 1 .
4. a weighting unit that derives a weighting value for each of the attribute information by updating a weighting value for each of the layers in accordance with a reference relationship of the attribute information between the sub-layers generated by the sub-layering unit; The information processing device according to claim 1 , further comprising:
5. a weighting unit that derives a weight value for each of the attribute information in accordance with a reference relationship of the attribute information between the hierarchies generated by the hierarchical generation unit and a reference relationship of the attribute information between the sub-hierarchies generated by the sub-hierarchical generation unit. The information processing device according to claim 1 , further comprising:
6. An encoding unit that encodes the attribute information and generates encoded data of the attribute information. The information processing device according to claim 1 , further comprising:
7. A generating unit that generates control information related to sub-hierarchical division of the attribute information, The encoding unit generates the encoded data including the control information generated by the generation unit. The information processing device according to claim 6.
8. The generating unit generates control information indicating whether or not sub-hierarchical division of the attribute information is permitted. The information processing device according to claim 7.
9. The generating unit generates control information indicating an interval at which the attribute information is sampled by the sub-hierarchical generation unit. The information processing device according to claim 7.
10. The generating unit generates control information indicating a method for sampling the attribute information by the sub-hierarchical generation unit. The information processing device according to claim 7.
11. hierarchically organizing attribute information of a point cloud that represents a three-dimensional object as a set of points, and generating a reference relationship between the attribute information between layers; In the generated hierarchy of the attribute information, the attribute information of the hierarchy is divided into sub-hierarchies, and a reference relationship of the attribute information between the sub-hierarchies is generated. Information processing methods.
12. a hierarchical generation unit that hierarchically generates attribute information of a point cloud that represents an object of a three-dimensional shape as a set of points, and generates a reference relationship between the attribute information between layers; a sub-hierarchy generation unit that generates sub-hierarchies of the attribute information in the hierarchy of the attribute information generated by the hierarchy generation unit and generates reference relationships of the attribute information between the sub-hierarchies; a de-hierarchicalization unit that de-hierarchizes the attribute information based on a reference relationship of the attribute information between the hierarchies generated by the hierarchicalization unit and a reference relationship of the attribute information between the sub-hierarchies generated by the sub-hierarchicalization unit; An information processing device comprising:
13. The sub-hierarchy generating unit arranges the attribute information of the processing target hierarchy in Morton order and samples the attribute information at equal intervals to generate a sub-hierarchy for the attribute information. The information processing device according to claim 12.
14. The sub-hierarchy generating unit arranges the attribute information of the processing target hierarchy in Morton order and samples the attribute information at non-equidistant intervals to generate sub-hierarchies of the attribute information. The information processing device according to claim 12.
15. a decoding unit that decodes the encoded data of the attribute information and restores the attribute information; The information processing device according to claim 12 , further comprising:
16. The decoding unit decodes the encoded data to restore control information related to sub-hierarchical division of the attribute information; The sub-hierarchical layer generation unit generates a sub-hierarchical layer for the attribute information based on the control information restored by the decoding unit. The information processing device according to claim 15.
17. The sub-layer generating unit generates a sub-layer of the attribute information based on control information indicating whether or not sub-layering of the attribute information restored by the decoding unit is permitted. The information processing device according to claim 16.
18. The sub-hierarchical layer generation unit generates a sub-hierarchical layer for the attribute information based on control information indicating a sampling interval of the attribute information restored by the decoding unit. The information processing device according to claim 16.
19. The sub-hierarchical layer generation unit generates a sub-hierarchical layer for the attribute information based on control information indicating a sampling method for the attribute information restored by the decoding unit. The information processing device according to claim 16.
20. hierarchically organizing attribute information of a point cloud that represents a three-dimensional object as a set of points, and generating a reference relationship between the attribute information between layers; In the generated hierarchy of the attribute information, the attribute information of the hierarchy is divided into sub-hierarchies, and a reference relationship of the attribute information between the sub-hierarchies is generated; The attribute information is inverted into a hierarchy based on the reference relationship of the attribute information between the generated hierarchies and the reference relationship of the attribute information between the generated sub-hierarchies. Information processing methods.
Citation Information
Patent Citations
Scalable point cloud compression with transform, and corresponding decompression
US20170347122A1
Point Cloud Compression
US20190080483A1
