Decoding processing device and method

Abstract

To be able to suppress the reduction of encoding efficiency.SOLUTION: Attribute information of a point cloud that expresses a three-dimensional shape object as a set of points is hierarchized, and a reference relationship is generated between the attribute information of the hierarchies. In the generated attribute information hierarchy, the attribute information in the hierarchy is subdivided into sub-hierarchies, and the reference relationships between the attribute information in the sub-hierarchies are generated. The present disclosure can be applied to, for example, an information processing device, an image processing system, an encoder, a decryption device, an electronic apparatus, an information processing method, or a program.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure provides: Decryption The present invention relates to a processing device and method, and in particular, to a method capable of suppressing a decrease in coding efficiency. Decryption The present invention relates to a processing device and method. [Background technology]

[0002] Conventionally, methods for encoding 3D data representing a three-dimensional structure, such as a point cloud, have been considered (see, for example, Non-Patent Document 1). Point cloud data consists of geometry data (also referred to as position information) and attribute data (also referred to as attribute information) for each point. Therefore, point cloud encoding is performed for both the geometry data and the attribute data. Various methods have been proposed as a method for encoding attribute data. For example, a technique called "Lifting" has been proposed (see, for example, Non-Patent Document 2). A method that enables scalable decoding of attribute data has also been proposed (see, for example, Non-Patent Document 3). Furthermore, a method that does not use Lifting but allows reference to decoded attribute data within LoD has also been considered (see, for example, Non-Patent Document 4). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] R. Mekuria, Student Member IEEE, K. Blom, P. Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video",tcsvt_paper_submitted_february.pdf [Non-patent document 2] Khaled Mammou, Alexis Tourapis, Jungsun Kim, Fabrice Robinet, Valery Valentin, Yeping Su, "Lifting Scheme for Lossy Attribute Encoding in TMC1", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42640, April 2018, San Diego, US [Non-patent document 3] Ohji Nakagami, Satoru Kuma, "[G-PCC] Spatial scalability support for G-PCC", ISO / IEC JTC1 / SC29 / WG11 MPEG2019 / m47352, March 2019, Geneva, CH [Non-patent document 4] Toshiyasu Sugio, "[G-PCC] Reference structure modification on attribute predicting transform in TMC13", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m46107, January 2019, Marrakech, MA Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the method described in Non-Patent Document 4, attribute data is sorted and encoded in Morton order within the LoD, so that the decoded attribute data that can be referenced is limited to that which precedes the attribute data to be decoded in the Morton order. In other words, there is a risk that the decoded attribute data will be limited to attribute data of points that are located in a biased direction in three-dimensional space with respect to the point of the attribute data to be decoded. As a result, there is a risk that the prediction accuracy of the attribute data to be processed, which is performed by referring to the decoded attribute data, will decrease, resulting in a decrease in encoding efficiency.

[0005] The present disclosure has been made in light of such circumstances, and makes it possible to suppress a decrease in coding efficiency. [Means for solving the problem]

[0006] One aspect of this technology Decryption The processing device generates a point cloud that represents a three-dimensional object as a set of points. a decoding unit that decodes encoded data to obtain geometry data and attribute data; a reverse-layering processing unit that reverse-layers the attribute data using the geometry data, generates sub-layers within the layer of the attribute data, and generates a reference relationship of the attribute data between the sub-layers; and a weighting unit that reflects the reference relationship between the sub-layers in a weight value of the attribute data for each layer. The information processing device is provided with:

[0007] One aspect of this technology Decryption The processing method is a decoding processing device, Point clouds represent 3D objects as a set of points. Decoding the encoded data to obtain geometry data and attribute data, inverting the hierarchy of the attribute data using the geometry data, generating sub-hierarchies within the hierarchy of the attribute data, generating reference relationships of the attribute data between the sub-hierarchies, and reflecting the reference relationships between the sub-hierarchies in weight values of the attribute data for each of the hierarchies. It is an information processing method.

[0010] One aspect of this technology Decryption The processing device and method include a point cloud that represents a three-dimensional object as a set of points. The encoded data is decoded to obtain geometry data and attribute data, the attribute data is then inversely layered using the geometry data, sub-layers are generated within the layer of the attribute data, reference relationships between the sub-layers of attribute data are generated, and the reference relationships between the sub-layers are reflected in the weight values of the attribute data for each layer. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 10 is a diagram illustrating an example of a reference relationship between attribute information according to the related art. [Figure 2]FIG. 10 is a diagram illustrating an example of a reference relationship according to Morton's order within a hierarchy. [Figure 3] FIG. 1 is a diagram illustrating a hierarchical method and an inverse hierarchical method for attribute data. [Figure 4] FIG. 10 is a diagram illustrating an example of how Sub LoD is generated. [Figure 5] FIG. 10 is a diagram illustrating an example of how Sub LoD is generated. [Figure 6] FIG. 10 is a diagram illustrating an example of a reference relationship. [Figure 7] FIG. 10 is a diagram illustrating an example of a reference relationship. [Figure 8] FIG. 1 is a block diagram illustrating an example of the main configuration of an encoding device. [Figure 9] FIG. 2 is a block diagram illustrating an example of the main configuration of an attribute information encoding unit. [Figure 10] FIG. 2 is a block diagram illustrating an example of the main configuration of a layering processing unit. [Figure 11] 10 is a flowchart illustrating an example of the flow of an encoding process. [Figure 12] 10 is a flowchart illustrating an example of the flow of an attribute information encoding process. [Figure 13] 10 is a flowchart illustrating an example of the flow of a layering process. [Figure 14] FIG. 2 is a block diagram illustrating an example of the main configuration of a decoding device. [Figure 15] FIG. 10 is a block diagram illustrating an example of the main configuration of an attribute information decoding unit. [Figure 16] FIG. 2 is a block diagram illustrating an example of the main configuration of a reverse layering processing unit. [Figure 17] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 18] 10 is a flowchart illustrating an example of the flow of an attribute information decoding process. [Figure 19] 10 is a flowchart illustrating an example of the flow of a layer inversion process. [Figure 20] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1.Generating Sub LoD 2. First embodiment (encoding device) 3. Second embodiment (decoding device) 4. Notes

[0014] <1. Sub LoD Generation> <References supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that were publicly known at the time of filing.

[0015] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: (mentioned above)

[0016] In other words, the contents described in the above-mentioned non-patent documents also serve as a basis for determining support requirements.

[0017] <Point Cloud> Previously, 3D data existed, such as point clouds, which represent three-dimensional structures using point group position information and attribute information, and meshes, which are composed of vertices, edges, and faces and define three-dimensional shapes using polygonal representations.

[0018] For example, in the case of a point cloud, a three-dimensional structure (a three-dimensional object) is represented as a collection of many points (a point cloud). In other words, point cloud data (also referred to as point cloud data) is composed of geometry data (also referred to as position information) and attribute data (also referred to as attribute information) for each point in the point cloud. The attribute data can include any information. For example, color information, reflectance information, normal information, etc. may be included in the attribute data. Therefore, the data structure is relatively simple, and by using a sufficient number of points, any three-dimensional structure can be represented with sufficient accuracy.

[0019] <Quantization of position information using voxels> Since such point cloud data has a relatively large amount of data, an encoding method using voxels was devised to compress the data volume through encoding, etc. A voxel is a three-dimensional region for quantizing geometry data (position information).

[0020] That is, the three-dimensional area containing the point cloud is divided into small three-dimensional areas called voxels, and each voxel indicates whether it contains a point. In this way, the position of each point is quantized in voxel units. Therefore, by converting point cloud data into such voxel data (also called voxel data), it is possible to suppress an increase in the amount of information (typically, to reduce the amount of information).

[0021] <octree> Furthermore, it was considered to construct an octree using such voxel data for geometry data. An octree is a tree structure of voxel data. The value of each bit in the lowest node of this octree indicates whether or not each voxel has a point. For example, a value of "1" indicates a voxel that contains a point, and a value of "0" indicates a voxel that does not contain a point. In an octree, one node corresponds to eight voxels. In other words, each node in the octree is made up of eight bits of data, and these eight bits indicate whether or not each of the eight voxels has a point.

[0022] The upper nodes of the Octree indicate whether or not there is a point in the area that combines the eight voxels corresponding to the lower nodes belonging to that node. In other words, the upper nodes are generated by combining the information of the voxels of the lower nodes. Note that if a node has a value of "0", that is, if none of the eight corresponding voxels contain a point, the node is deleted.

[0023] By doing this, a tree structure (octree) consisting of nodes whose values are not "0" is constructed. In other words, the octree can indicate whether or not there are points at each voxel resolution. By encoding the position information as an octree, the point cloud data at that resolution can be restored by decoding it from the highest resolution (top layer) to the desired layer (resolution). In other words, it is possible to easily decode at any resolution without decoding information at unnecessary layers (resolutions). In other words, it is possible to achieve scalability of voxels (resolutions).

[0024] Furthermore, by omitting nodes with a value of "0" as described above, it is possible to lower the resolution of voxels in areas where no points exist, thereby further suppressing the increase in the amount of information (typically reducing the amount of information).

[0025] <lifting> In contrast, when encoding attribute data (attribute information), the geometry data (position information) is assumed to be known, including degradation due to encoding, and encoding is performed using the positional relationships between points. As a method for encoding such attribute data, methods using a transformation called RAHT (Region Adaptive Hierarchical Transform) or Lifting as described in Non-Patent Document 2 have been considered. By applying these technologies, attribute data can also be hierarchically organized, like the octree of geometry data.

[0026] For example, in the case of Lifting described in Non-Patent Document 2, attribute data of each point is coded as a difference value from a predicted value derived using attribute data of other points. At this time, each point is hierarchically organized, and the difference value is derived according to the hierarchical structure.

[0027] That is, for the attribute data for each point, each point is classified into a prediction point and a reference point, the attribute data of the reference point is used to derive a predicted value of the attribute data of the prediction point, and the difference between the attribute data of the prediction point and the predicted value is derived. By repeating this process recursively for the reference points, the attribute data of each point is hierarchically organized.

[0028] However, this hierarchical structure is generated independently of the hierarchical structure of the geometry data (e.g., Octree), and does not fundamentally correspond to the hierarchical structure of the geometry data. In order to restore point cloud data, it is necessary to associate the geometry data with the attribute data, which requires decoding the geometry data and attribute data up to the highest resolution (i.e., the lowest layer). In other words, the method applying Lifting described in Non-Patent Document 2 does not support resolution scalable decoding.

[0029] Layering for scalable decoding In contrast, the layering described in Non-Patent Document 3 supports resolution-scalable decoding. In the method described in Non-Patent Document 3, attribute data is layered to match the hierarchical structure of the octree of the geometry data. In other words, if a point exists in an area corresponding to a voxel of the geometry data (attribute data corresponding to that point exists), a reference point and a prediction point are selected so that a point also exists in a voxel one layer above that voxel (attribute data corresponding to that point exists). In other words, attribute information is layered according to the hierarchical structure of the octree of the geometry data.

[0030] By associating the hierarchical structure of attribute data with the hierarchical structure of geometry data, it is possible to easily restore point cloud data of a desired resolution without decoding down to the lowest layer. In this way, the method applying the technology described in Non-Patent Document 3 supports resolution scalable decoding.

[0031] <Reference relationship within the hierarchy> In the above-described method, a reference relationship of attribute information is generated between layers, which means that attribute information in a layer to be processed can be predicted by referencing attribute information in other layers.

[0032] In contrast, in the method described in Non-Patent Document 4, attribute information in the layer to be processed can be referenced if it has already been decoded. For example, the layered attribute data (Attribute Lod) 1 shown in A of Fig. 1 is obtained by classifying and layering attribute data 2 for each point, as shown in B of Fig. 1. In B of Fig. 1, the vertical direction indicates the layer (LoD (Level Of Detail)), and each circle indicates attribute data 2 for each point. In B of Fig. 1, only one circle is assigned a symbol, but all the circles shown in B of Fig. 1 are attribute data 2 for each point.

[0033] Decoding is performed sequentially, layer by layer, starting from the highest layer. Therefore, when decoding attribute data 2A for a certain point, attribute data 2 for each point in the higher layer, indicated by a white circle, has already been decoded, so it is possible to refer to them.

[0034] Furthermore, within a layer, the attribute data 2 for each point is sorted in Morton order and decoded in that order. In B of Fig. 1, the attribute data 2 for each point in each layer is sorted in Morton order. In the figure, the horizontal direction from left to right corresponds to Morton order.

[0035] In other words, when the attribute data 2A of a certain point is decoded, the attribute data 2B of each point located to the left of it has already been decoded and can be referenced.

[0036] By sorting in Morton order, the attribute data 2 for each point is scanned for each three-dimensional partial region obtained by dividing the three-dimensional space. Therefore, a directionality (bias) of the three-dimensional spatial position occurs in the processing order.

[0037] For example, assume that each point of the point cloud data exists within the three-dimensional region 3 in FIG. 2. As described above, in B of FIG. 1, the decoded attribute data 2B for each point is located before the attribute data 2A for a certain point in Morton's order. Therefore, in the three-dimensional region 3 in FIG. 2, the decoded attribute data 2B for each point shown in B of FIG. 1 exists within a partial region 3B located in a predetermined direction as viewed from the attribute data 2A for a certain point. In other words, when the position of the attribute data 2A for a certain point is used as a reference, a bias occurs in the relative position (direction) of the decoded attribute data 2B for each point.

[0038] When processing is performed according to Morton order, such bias occurs in the reference direction during prediction, and this bias may reduce prediction accuracy and coding efficiency.

[0039] <Sub-hierarchy> Therefore, for example, as described in "Method 1" in the top row of the table shown in Figure 3, when hierarchizing attribute data (attribute information), a sub-hierarchy (Sub LoD) is further generated within the hierarchy (LoD), and a reference relationship of attribute data between the sub-hierarchies is generated (i.e., a reference relationship of attribute data within the hierarchy (LoD) is generated).

[0040] For example, the attribute data (attribute information) of a point cloud, which represents a three-dimensional object as a collection of points, is used to generate a reference relationship between the attribute data at its hierarchical levels, and then, within the generated attribute data hierarchy, the attribute data at that hierarchical level is sub-hierarchized, and a reference relationship between the attribute data at its sub-hierarchical levels is generated.

[0041] For example, an information processing device may include a layering unit that layers attribute data (attribute information) of a point cloud that represents a three-dimensional object as a collection of points and generates a reference relationship of attribute data between those layers, and a sub-layering unit that sub-layers the attribute data of the layer of attribute data generated by the layering unit and generates a reference relationship of attribute data between those sub-layers.

[0042] In this way, it is possible to generate a reference relationship of attribute data within a layer so as to suppress bias in the prediction direction, and it is possible to suppress a decrease in coding efficiency.

[0043] In this case, any technique can be used to layer the attribute data. For example, Lifting, which is not compatible with scalable decoding and is described in Non-Patent Document 2, may be used. Alternatively, a layering technique compatible with scalable decoding and described in Non-Patent Document 3 may be applied. Of course, other techniques may also be used.

[0044] The method for creating sub-hierarchies is also arbitrary. For example, as described in "Method 1-1" in the second row from the top of the table in Figure 3, a process of sampling (selecting) a portion of the attribute data for each point in a hierarchy to create a sub-hierarchy may be repeated recursively. In other words, a sub-hierarchy may be generated by sampling a portion from the group of nodes in the hierarchy (LoD) to be processed in the hierarchical structure of the attribute data.

[0045] For example, as described in "Method 1-1-1" in the third row from the top of the table shown in Figure 3, the attribute data of the hierarchy to be processed may be arranged in Morton order, and a portion of it may be sampled at equal intervals to generate a sub-hierarchy.

[0046] For example, as in the case of B in Figure 1, for hierarchical attribute data, the attribute data for each point in the hierarchy to be sub-hierarchicalized is sorted in Morton order. Then, the sorted attribute data for each point is sampled at equal intervals. Figure 4 shows an example in which every third point, i.e., every fourth point, is sampled (LodUniformQuant = 4). LodUniformQuant is a syntax element that indicates the sampling interval. LodUniformQuant = 4 indicates that every fourth point in the string of attribute data for each point sorted in Morton order is sampled. In other words, when one piece of attribute data is sampled, the fourth attribute data in Morton order is sampled from there.

[0047] In Fig. 4A, the attribute data 12 for each point, represented by a circle, is arranged in Morton order, and one out of every four pieces of attribute data 12 (represented by a gray circle) is sampled. In other words, as shown in Fig. 4B, a lowest sub-level (Sub LoD) is generated, consisting of the attribute data 12 for each point, represented by a white circle, that has not been sampled.

[0048] Similarly, one out of every four pieces of attribute data 12 for each point (point attribute data 12 indicated by black circles) is sampled from the row of sampled point attribute data 12. That is, as shown in Fig. 4C, the second-lowest sub-layer (Sub LoD) is generated, which is made up of the point attribute data 12 indicated by gray circles that were not sampled.

[0049] Similarly, one out of every four pieces of attribute data 12 for each point (the attribute data 12 for each point indicated by a diagonally shaded circle) is sampled from the row of sampled attribute data 12 for each point. That is, as shown in Fig. 4D, a third sub-hierarchy from the bottom is generated, which is made up of the attribute data 12 for each point indicated by a black circle that was not sampled.

[0050] In this case, since the attribute data 12 for each point is sampled, a top-level sub-hierarchy made up of the attribute data 12 for each sampled point is also generated (D in FIG. 4).

[0051] A reference relationship is also created between these sub-layers. That is, attribute data in higher sub-layers is decoded first, making it possible to refer to attribute data in higher sub-layers.

[0052] In this case, since the entire string of attribute data 12 for each point is sampled to generate sub-hierarchies, there is less bias in the positions of those points in three-dimensional space. In other words, compared to the method described in Non-Patent Document 4, it is possible to suppress bias in the reference direction of attribute data. Therefore, it is possible to suppress a decrease in prediction accuracy and a decrease in coding efficiency.

[0053] In this case, the attribute data 12 for each point is also sorted in Morton order, so that it is possible to refer to attribute data that are relatively close in three-dimensional space. This makes it possible to suppress a decrease in prediction accuracy and a decrease in coding efficiency compared to when attribute data are selected randomly.

[0054] It should be noted that it may be possible to refer not only to attribute data in a higher sub-layer, but also to decoded attribute data in the same sub-layer as the attribute data to be processed.

[0055] 5 shows attribute data 12 for each point in a certain layer sorted in the order in which they are decoded. In this case, if attribute data 12A for a certain point is to be decoded, attribute data 12 for each point within the range indicated by the double-headed arrow 12B (i.e., attribute data 12 for each point located to the left of attribute data 12A for the certain point) has already been decoded and can be referenced. This increases the number of candidates for reference points, thereby making it possible to suppress a decrease in prediction accuracy and a decrease in coding efficiency.

[0056] In the above, we have explained that attribute data is sampled at equal intervals when generating sub-hierarchies, but this is not a limitation and sampled at non-equidistant intervals are also possible. In other words, as described in "Method 1-1-2" in the fourth row from the top of the table in Figure 3, sub-hierarchies can be generated by arranging the attribute data of the target hierarchy in Morton order and then sampling some of the data at non-equidistant intervals.

[0057] For example, you can sample attribute data in a sub-hierarchy at non-uniform intervals (i.e., not at regular intervals), for example, alternating between every third and every third (LodUniformQuant = 2 or 3).

[0058] Also, for example, the sampling interval may be changed for each sub-layer. For example, the lowest sub-layer may be sampled every third sub-layer (LodUniformQuant = 2), and the sub-layers above that may be sampled every fourth sub-layer (LodUniformQuant = 4). In other words, the sampling interval may be changed depending on the depth of the sub-layer.

[0059] By doing so, it is possible to set the sampling interval according to, for example, the resolution, and it is possible to suppress, for example, a decrease in image quality and a decrease in coding efficiency.

[0060] <weight value> Note that such sub-hierarchy may be reflected in the weight value of the attribute data. In conventional methods, attribute data for each point is weighted based on the reference relationship between layers (LoDs) (e.g., reference distance, reference number, etc.). When sub-hierarchy is performed as described above, the reference relationship between the sub-layers (e.g., reference distance, reference number, etc.) may also be reflected in the weight value. In other words, as described in "Method 1-2" in the fifth row from the top of the table shown in FIG. 3, the weight value may be derived using the reference relationship between sub-layers (Sub LoDs).

[0061] For example, when attribute data is layered (LoD) using Lifting as described in Non-Patent Document 2, a weight value is set for each attribute data based on the reference relationship. Similarly, when attribute data layered using this weight value setting method is sub-layered as described above, the reference relationship between the sub-layers is reflected in the weight value set for each attribute data. That is, the weight value for each attribute data is derived using the reference relationship between layers and sub-layers established as indicated by the arrows in FIG. 6. This allows the reference relationship between sub-layers to be reflected in the weight value between layers generated by a method that does not support scalable decoding.

[0062] For example, when attribute data is layered (LoD) using a method compatible with scalable decoding described in Non-Patent Document 3, a fixed weight value (weight value for each layer) is assigned to each attribute data for each layer (LoD). When attribute data layered using this method of setting weight values is sub-layered as described above, the weight value for each layer reflects the reference relationship between sub-layers, as described in "Method 1-2-1" in the sixth row from the top of the table shown in Figure 3. In other words, the weight value for each attribute data is derived by updating the weight value for each layer using the reference relationship between sub-layers as indicated by the arrows in A of Figure 7.

[0063] For example, a coefficient α may be set according to the number of references between sub-layers, and the weight value w1 for each layer may be updated by multiplying the coefficient α by the weight value w1 for each layer, as shown in B of Figure 7.

[0064] Furthermore, when updating the weight value w1 of a certain layer, the weight value w2 of another layer may be updated to correspond to the updated weight value w1. For example, as shown in C of Fig. 7, when multiplying the weight value w1 by a coefficient α, the weight value w2 may be multiplied by (1-α).

[0065] This allows the reference relationships between sub-layers to be reflected in the weights between layers generated by a method that does not support scalable decoding, thereby further reducing the reduction in coding efficiency.

[0066] <Control information signaling> As described in "Method 1-3" in the seventh row from the top of the table shown in Fig. 3, control information regarding the sub-hierarchical division of such attribute data may be signaled. The content of this control may be any.

[0067] <sub_lod_enable_flag> For example, control information indicating whether sub-layering of attribute data is permitted may be signaled (transmitted from the encoding side to the decoding side). This control information indicating whether sub-layering is permitted is flag information (e.g., sub_lod_enable_flag) indicating whether sub-layering can be performed in the target data unit. For example, if this sub_lod_enable_flag is true (e.g., "1"), this indicates that sub-layering can be performed. Also, if sub_lod_enable_flag is false (e.g., "0"), this indicates that sub-layering cannot be performed.

[0068] If this sub_lod_enable_flag is not signaled (transmitted), the decoding side may consider the sub_lod_enable_flag to be false. Conversely, if this sub_lod_enable_flag is not signaled (transmitted), the decoding side may consider the sub_lod_enable_flag to be true.

[0069] If sub_lod_enable_flag is false, sub-layering is not permitted, and therefore signaling of other control information related to sub-layering for that data unit can be omitted. This makes it possible to suppress a decrease in coding efficiency. In other words, on the decoding side, parsing of other control information related to sub-layering for that data unit can be omitted. In other words, on the decoding side, it is only necessary to parse other control information related to sub-layering for only data units for which sub_lod_enable_flag is true. This makes it possible to suppress an increase in the load of the decoding process.

[0070] As described above, the control information indicating whether sub-layering is permitted can also be considered as control information indicating whether other control information related to sub-layering for that data unit is signaled.Furthermore, the control information indicating whether sub-layering is permitted can also be considered as control information indicating whether sub-layering is prohibited.

[0071] The control information indicating whether sub-layering is permitted can be applied to any data unit, and may be signaled for each data unit. For example, attribute data may be used as the target data unit, and sub_lod_enable_flag may be signaled for each attribute data. Also, for example, multiple sub_lod_enable_flag may be transmitted together for each data unit (e.g., each sequence) higher than the attribute data.

[0072] <sub_lod_distance> For example, control information that controls the sampling interval (or sampling period) when creating a sub-layer may be signaled. This control information that controls the sampling interval is information (e.g., sub_lod_distance) that indicates the interval at which sampling should be performed when creating a sub-layer for the target data unit. For example, if sub_lod_distance = 2, this indicates that the sampling period is 2, that is, one out of every two nodes is sampled (sampled every other node). In other words, if sub_lod_distance > 2, sampling is performed at equal intervals indicated by sub_lod_distance, and a sub-layer is generated.

[0073] Note that sub_lod_distance = 0 indicates that no sub-layering is performed (no sub-layer (Sub LoD) is formed). In other words, sub_lod_distance can be said to be control information that indicates whether or not sub-layering is performed.

[0074] Also, if this sub_lod_distance is not signaled (transmitted), it may be assumed that sub_lod_distance=0 on the decoding side.

[0075] Furthermore, the sampling intervals may be non-uniform. For example, multiple sub_lod_distances may be signaled for the same data unit, allowing multiple intervals (cycles) to be assigned. For example, the sub_lod_distance values may be set to "2" and "3," and these intervals may be adopted sequentially, such as 2 → 3 → 2 → 3, thereby realizing non-uniform intervals. Furthermore, the sampling intervals may be represented by the identification number of a function or table, etc.

[0076] The control information for controlling the sampling interval during this sub-layering can target any data unit and may be signaled for each arbitrary data unit. For example, the layer (LoD) of attribute data may be the target data unit, and sub_lod_distance may be signaled for each layer of attribute data.

[0077] <sub_lod_mode> For example, control information (eg, sub_lod_mode) indicating the sampling method for sub-layering may be signaled.

[0078] For example, sub_lod_mode = 0 indicates that sampling is performed at the same intervals in all sub-hierarchies. In other words, in this case, sampling is performed at one type of interval in all sub-hierarchies.

[0079] Furthermore, sub_lod_mode = 1 indicates that the sampling intervals are pattern 1 (unequal intervals). Similarly, sub_lod_mode = 2 indicates that the sampling intervals are pattern 2 (unequal intervals), which is different from pattern 1.

[0080] By sampling at non-uniform intervals, it is possible to change the sampling interval depending on the depth of the sub-level (SubLod), for example. It is also possible to change the sampling interval within a sub-level. In other words, the above-mentioned Pattern 1 and Pattern 2 show the sampling interval patterns for all sub-levels. The pattern distance can be overridden by the header, etc.

[0081] <Other> Note that the control information related to sub-layering is not limited to the above example. Multiple pieces of control information may be applied. For example, both sub_lod_distance and sub_lod_mode may be applied. For example, sub_lod_enable_flag may be applied together with sub_lod_distance, sub_lod_mode, or both.

[0082] <Reverse hierarchy> Attribute data that has been hierarchically organized and sub-hierarchically organized as described above may be inverted into a hierarchical structure by utilizing the reference relationships between sub-levels (Sub LoDs), as described in "Method 2" in the eighth row from the top of the table in Figure 3.

[0083] Furthermore, for example, as described in "Method 2-1" in the ninth row from the top of the table shown in Figure 3, sub-hierarchies (Sub LoDs) may be generated in the same manner as in the above-mentioned "Method 1," and the reference relationships between the generated sub-hierarchies may be used to perform inverse hierarchical construction.

[0084] For example, the attribute data (attribute information) of a point cloud that represents a three-dimensional object as a collection of points may be hierarchized, attribute data reference relationships between those hierarchies may be generated, the attribute data may be subhierarchized within the hierarchy, attribute data reference relationships between those subhierarchies may be generated, and the attribute data may be reverse-hierarchized based on the generated attribute data reference relationships between the hierarchies and the generated attribute data reference relationships between the subhierarchies.

[0085] For example, an information processing device may include a layering unit that layers attribute data (attribute information) of a point cloud that represents a three-dimensional object as a collection of points and generates a reference relationship of the attribute data between the layers, a sub-layering unit that sub-layers the attribute data within the layer generated by the layering unit and generates a reference relationship of the attribute data between the sub-layers, and a de-layering unit that de-layers the attribute data based on the reference relationship of the attribute data between the layers generated by the layering unit and the reference relationship of the attribute data between the sub-layers generated by the sub-layering unit.

[0086] By doing this, attribute data that has been layered or sub-layered by applying the above-mentioned "Method 1" or the like can be correctly de-layered, thereby suppressing a decrease in coding efficiency.

[0087] In creating the sub-hierarchy, for example, "Method 1-1-1" in the third row from the top of the table shown in Fig. 3 may be applied. In other words, the attribute data of the hierarchy to be processed may be arranged in Morton order, and a portion of this data may be sampled at equal intervals to generate the sub-hierarchy.

[0088] Furthermore, in creating the sub-hierarchy, for example, "Method 1-1-2" in the fourth row from the top of the table shown in Fig. 3 may be applied. In other words, the attribute data of the hierarchy to be processed may be arranged in Morton order, and some of the data may be sampled at non-uniform intervals to generate the sub-hierarchy.

[0089] Furthermore, in the sub-layering, for example, as described in "Method 2-2" in the tenth row from the top of the table shown in Figure 3, sub-layers (Sub LoD) may be generated based on control information regarding the sub-layering of attribute data signaled as in the above-mentioned "Method 1-3," and reverse layering may be performed using the reference relationships between the generated sub-layers.

[0090] For example, the attribute data may be divided into sub-layers based on the control information (for example, sub_lod_enable_flag) indicating whether or not the above-described sub-layering of the attribute data is permitted.

[0091] Furthermore, for example, the attribute data may be divided into sub-hierarchies based on control information (for example, sub_lod_distance) indicating the sampling interval of the attribute data described above.

[0092] Furthermore, for example, the attribute data may be divided into sub-hierarchies based on control information (for example, sub_lod_mode) that indicates the sampling method of the attribute data described above.

[0093] Of course, attribute data may be divided into sub-hierarchies based on control information other than these, or may be divided into sub-hierarchies based on a plurality of control information.

[0094] By performing the inverse layering as described above, sub-layered attribute data can be inversely layered more accurately, thereby suppressing bias in the prediction direction and reducing the reduction in coding efficiency.

[0095] 2. First Embodiment <Encoding device> Next, a device to which the present technology described above in <1. Generation of Sub LoD> is applied will be described. Fig. 8 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology is applied. The encoding device 100 shown in Fig. 8 is a device that encodes a point cloud (3D data). The encoding device 100 encodes the point cloud by applying the present technology described above in <1. Generation of Sub LoD>.

[0096] Note that Fig. 8 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 8. In other words, in encoding device 100, there may be processing units that are not shown as blocks in Fig. 8, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 8.

[0097] As shown in FIG. 8, the encoding device 100 includes a position information encoding unit 101, a position information decoding unit 102, a point cloud generating unit 103, an attribute information encoding unit 104, and a bitstream generating unit 105.

[0098] The position information encoding unit 101 encodes geometry data (position information) of a point cloud (3D data) input to the encoding device 100. Any encoding method may be used as long as it is compatible with scalable decoding. For example, the position information encoding unit 101 layers the geometry data to generate an octree and encodes the octree. Furthermore, processing such as filtering and quantization for noise suppression (denoising) may also be performed. The position information encoding unit 101 supplies the generated encoded data of the geometry data to the position information decoding unit 102 and the bitstream generation unit 105.

[0099] The position information decoding unit 102 acquires the coded data of the geometry data supplied from the position information encoding unit 101 and decodes the coded data. This decoding method may be any method compatible with the coding performed by the position information encoding unit 101. For example, processing such as filtering or inverse quantization for denoising may be performed. The position information decoding unit 102 supplies the generated geometry data (decoded result) to the point cloud generation unit 103.

[0100] The point cloud generation unit 103 acquires attribute data (attribute information) of the point cloud input to the encoding device 100 and geometry data (decoded result) supplied from the position information decoding unit 102. The point cloud generation unit 103 performs processing (recolor processing) to match the attribute data with the geometry data (decoded result). The point cloud generation unit 103 supplies the attribute data associated with the geometry data (decoded result) to the attribute information encoding unit 104.

[0101] The attribute information encoding unit 104 acquires the geometry data (decoded result) and attribute data supplied from the point cloud generation unit 103. The attribute information encoding unit 104 uses the geometry data (decoded result) to encode the attribute data and generate encoded data of the attribute data.

[0102] In this case, the attribute information encoding unit 104 applies the present technology described above in <1. Generation of Sub LoD> to encode the attribute data. For example, when layering the attribute data (attribute information), the attribute information encoding unit 104 generates a sub-layer (Sub LoD) within the layer (LoD) and generates a reference relationship of the attribute data between the sub-layers. The attribute information encoding unit 104 supplies the generated coded data of the attribute data to the bitstream generation unit 105.

[0103] The bitstream generation unit 105 obtains coded data of geometry data supplied from the position information encoding unit 101. The bitstream generation unit 105 also obtains coded data of attribute data supplied from the attribute information encoding unit 104. The bitstream generation unit 105 generates a bitstream including these coded data. The bitstream generation unit 105 outputs the generated bitstream to the outside of the encoding device 100.

[0104] With this configuration, the encoding device 100 can generate sub-layers within a layer, generate reference relationships of attribute data between the sub-layers, and encode the attribute data using the reference relationships. This makes it possible to generate reference relationships of attribute data within a layer so as to suppress bias in the prediction direction, and to suppress a decrease in encoding efficiency.

[0105] Each of these processing units (position information encoding unit 101 to bitstream generation unit 105) of the encoding device 100 may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Furthermore, each processing unit may have, for example, a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may implement part of the above-described processing using a logic circuit and other parts by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may implement part of the above-described processing using a logic circuit, other processing units may implement the above-described processing by executing a program, and still other processing units may implement the above-described processing by both a logic circuit and by executing a program.

[0106] <Attribute information encoding section> Fig. 9 is a block diagram showing an example of the main configuration of the attribute information encoding unit 104 (Fig. 8). Note that Fig. 9 shows the main processing units, data flows, etc., and is not limited to what is shown in Fig. 9. In other words, the attribute information encoding unit 104 may include processing units not shown as blocks in Fig. 9, and may include processing and data flows not shown as arrows, etc. in Fig. 9.

[0107] As shown in FIG. 8, the attribute information encoding unit 104 includes a layering processing unit 111, a quantization unit 112, and an encoding unit 113.

[0108] The layering processing unit 111 performs processing related to layering of attribute data. For example, the layering processing unit 111 acquires attribute data and geometry data (decoded results) supplied from the point cloud generation unit 103. The layering processing unit 111 layers the attribute data using the geometry data. At this time, the layering processing unit 111 performs layering by applying the present technology described above in <1. Generation of Sub LoD>. For example, in layering the attribute data (attribute information), the layering processing unit 111 further generates sub-layers (Sub LoDs) within a layer (LoD) and generates reference relationships of attribute data between the sub-layers. The layering processing unit 111 supplies the layered attribute data (difference values) to the quantization unit 112.

[0109] At this time, the layering processing unit 111 can also generate control information related to sub-layering. In this case, the layering processing unit 111 supplies the generated control information to the quantization unit 112 together with the attribute data (difference value).

[0110] The quantization unit 112 acquires the attribute data supplied from the layering processing unit 111 (and the control information if the control information is generated in the layering processing unit 111). The quantization unit 112 quantizes the attribute data. Any quantization method may be used. The quantization unit 112 supplies the quantized attribute data (and the control information) to the encoding unit 113.

[0111] The encoding unit 113 acquires the quantized attribute data (and control information) supplied from the quantization unit 112. The encoding unit 113 encodes the attribute data to generate encoded data of the attribute data. Any encoding method may be used. Furthermore, if control information related to sub-layering is supplied, the encoding unit 113 includes the control information in the generated encoded data. In other words, the encoding unit 113 generates encoded data of the attribute data including the control information related to sub-layering. The encoding unit 113 supplies the generated encoded data to the bitstream generation unit 105.

[0112] By performing layering as described above, the attribute information encoding unit 104 can generate sub-layers within a layer, generate reference relationships of attribute data between the sub-layers, and encode the attribute data using the reference relationships. This makes it possible to generate reference relationships of attribute data within a layer so as to suppress bias in the prediction direction, and to suppress a decrease in encoding efficiency.

[0113] These processing units (layering processing unit 111 to encoding unit 113) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Furthermore, each processing unit may have, for example, a CPU, ROM, RAM, etc., and may execute a program using these to realize the above-mentioned processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-mentioned processing using a logic circuit and the other part by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may realize part of the above-mentioned processing using a logic circuit, other processing units may execute a program to realize the above-mentioned processing, and still other processing units may realize the above-mentioned processing using both a logic circuit and by executing a program.

[0114] <Hierarchical Processing Unit> Fig. 10 is a block diagram showing an example of the main configuration of the layering processing unit 111 (Fig. 9). Note that Fig. 10 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, the layering processing unit 111 may include processing units that are not shown as blocks in Fig. 10, and may include processing and data flows that are not shown as arrows, etc. in Fig. 10.

[0115] As shown in FIG. 10, the layering processing unit 111 includes a control unit 121, a layering processing unit 122, a Sub LoD generating unit 123, an inverting unit 124, and a weighting unit 125.

[0116] The control unit 121 performs processing related to the control of layering and sub-layering. For example, the control unit 121 acquires attribute data and geometry data (decoded results) supplied from the point cloud generation unit 103. The control unit 121 supplies the acquired attribute data and geometry data (decoded results) to the layering processing unit 122.

[0117] The control unit 121 also controls the layering processing unit 122 and the Sub LoD generation unit 123 to perform layering and sub-layering. For example, the control unit 121 applies the present technology described above in <1. Sub LoD> to perform layering and sub-layering. For example, the control unit 121 causes sub-layering to be performed for desired layers. That is, the control unit 121 can cause sub-layering to be performed for some layers or for all layers.

[0118] The control unit 121 can also generate control information related to the sub-layering of the attribute data. For example, the control unit 121 can generate the various syntax elements (e.g., sub_lod_enable_flag, sub_lod_distance, sub_lod_mode, etc.) described above in <1. Generation of Sub LoD> as control information. The control unit 121 supplies the control information thus generated to the quantization unit 112, which then transmits the control information to the decoding side.

[0119] The layering processing unit 122 performs processing related to layering of attribute data. For example, the layering processing unit 122 acquires attribute data and geometry data (decoding results) supplied from the control unit 121.

[0120] The layering processing unit 122 layers the acquired attribute data using the acquired geometry data under the control of the control unit 121. Any layering method may be used. For example, it may be a method that does not support scalable decoding, such as Lifting described in Non-Patent Document 2, or a method that supports scalable decoding, such as described in Non-Patent Document 3. The layering processing unit 122 supplies the layered attribute data and geometry data to the Sub LoD generation unit 123.

[0121] The Sub LoD generation unit 123 performs processing related to sub-layering. For example, the Sub LoD generation unit 123 acquires layered attribute data and geometry data supplied from the layering processing unit 122.

[0122] The Sub LoD generation unit 123 generates a sub-hierarchy within the hierarchy of attribute data under the control of the control unit 121. The Sub LoD generation unit 123 performs this sub-hierarchy generation using geometry data. In other words, the Sub LoD generation unit 123 generates a reference relationship of attribute data within a hierarchy (LoD) as in "Method 1" in the top row of the table shown in Fig. 3 described above in <1. Generation of Sub LoD>.

[0123] In this way, the Sub LoD generation unit 123 can generate a reference relationship of attribute data within a layer so as to suppress bias in the prediction direction, and can suppress a decrease in coding efficiency.

[0124] This generation method is arbitrary. For example, the Sub LoD generation unit 123 may recursively repeat the process of sampling (selecting) part of the attribute data for each point in a layer to create a sub-layer, as in "Method 1-1" in the second row from the top of the table shown in Fig. 3.

[0125] Alternatively, the Sub LoD generation unit 123 may generate a sub-hierarchy by arranging the attribute data of the hierarchy to be processed in Morton order and sampling a portion of the data at equal intervals, as in "Method 1-1-1" in the third row from the top of the table shown in Figure 3.

[0126] Alternatively, the Sub LoD generation unit 123 may generate a sub-hierarchy by arranging the attribute data of the hierarchy to be processed in Morton order and sampling some of the data at non-uniform intervals, as in "Method 1-1-2" in the fourth row from the top of the table shown in Figure 3.

[0127] Of course, other methods may be used. Note that if sub-layering is not permitted by the control unit 121, the Sub LoD generation unit 123 can omit sub-layering. In other words, the Sub LoD generation unit 123 can perform sub-layering only on layers permitted by the control unit 121.

[0128] The Sub LoD generating unit 123 supplies the attribute data thus sub-hierarchized under the control of the control unit 121 to the inverting unit 124 .

[0129] The inversion unit 124 performs processing related to layer inversion. For example, the inversion unit 124 acquires attribute data supplied from the Sub LoD generation unit 123. In this attribute data, information on each layer is layered in the order in which it was generated.

[0130] The inversion unit 124 inverts the layers of the attribute data. For example, the inversion unit 124 assigns a layer number (a number for identifying a layer where the value is 0 for the top layer, incremented by 1 for each layer down, and the lowest layer has the maximum value) to each layer of the attribute data in the reverse order of the generation order. In other words, by assigning this layer number to the data of each layer, the layers are generated in order from the lowest layer to the top layer.

[0131] The inverting unit 124 supplies the attribute data with the inverted hierarchy to the weighting unit 125 .

[0132] The weighting unit 125 performs processing related to weighting. For example, the weighting unit 125 acquires attribute data supplied from the inverting unit 124. The weighting unit 125 derives a weight value for the acquired attribute data. Note that the method for deriving this weight value is arbitrary.

[0133] For example, the weighting unit 125 may derive weight values using reference relationships between sub-hierarchies (Sub LoDs), as in "Method 1-2" in the fifth row from the top of the table shown in Figure 3 described above in <1. Generation of Sub LoDs>.

[0134] Furthermore, when attribute data is layered (LoD) using Lifting described in Non-Patent Document 2, the weighting unit 125 may reflect the reference relationship of attribute data between sub-layers in the weight value set for each attribute data in the layering. In other words, the weighting unit 125 may derive the weight value according to both the reference relationship of attribute data between layers and the reference relationship of attribute data between sub-layers.

[0135] Furthermore, when attribute data is layered (LoD) using a method compatible with scalable decoding described in Non-Patent Document 3, the weighting unit 125 may reflect the reference relationship of attribute data between sub-layers in the weight value for each layer generated in the layering, as in "Method 1-2-1" in the sixth row from the top of the table shown in Figure 3.

[0136] Of course, other methods may be used. Note that the weighting unit 125 may also omit this weighting.

[0137] The weighting unit 125 predicts the attribute data of each point using the reference relationship between the weight values and the attribute data derived as described above, and derives a predicted value. The weighting unit 125 also derives a difference value between the attribute data of each point and its predicted value.

[0138] The weighting unit 125 supplies the derived attribute data (difference value) to the quantization unit 112 (FIG. 9). The weighting unit 125 may also supply the derived weight value as control information to the quantization unit 112, which may then transmit the control information to the decoding side.

[0139] These processing units (control unit 121 to weighting unit 125) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, ROM, RAM, etc., and may execute a program using these to realize the above-mentioned processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-mentioned processing using a logic circuit and other parts by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may realize part of the above-mentioned processing using a logic circuit, other processing units may execute a program to realize the above-mentioned processing, and still other processing units may realize the above-mentioned processing using both a logic circuit and by executing a program.

[0140] <Encoding process flow> Next, a description will be given of the processing executed by the encoding device 100. The encoding device 100 encodes point cloud data by executing an encoding process. An example of the flow of this encoding process will be described with reference to the flowchart in FIG.

[0141] When the encoding process starts, in step S101, the position information encoding unit 101 of the encoding device 100 encodes the geometry data (position information) of the input point cloud, and generates encoded data of the geometry data.

[0142] In step S102, the position information decoding unit 102 decodes the coded data of the geometry data generated in step S101 to generate position information.

[0143] In step S103, the point cloud generation unit 103 performs recolor processing using the attribute data (attribute information) of the input point cloud and the geometry data (decoding result) generated in step S102, and associates the attribute data with the geometry data.

[0144] In step S104, the attribute information encoding unit 104 performs attribute information encoding processing to encode the attribute data recolored in step S103 and generate encoded data of the attribute data. At this time, the attribute information encoding unit 104 performs processing by applying the present technology described above in <1. Generation of Sub LoD>. Details of the attribute information encoding processing will be described later.

[0145] In step S105, the bitstream generating unit 105 generates and outputs a bitstream including the coded data of the geometry data generated in step S101 and the coded data of the attribute data generated in step S104.

[0146] When the process of step S105 is completed, the encoding process ends.

[0147] By performing the processing of each step in this manner, the encoding device 100 can generate a reference relationship of attribute data within a hierarchy so as to suppress bias in the prediction direction, and can suppress a decrease in encoding efficiency.

[0148] <Attribute information encoding process flow> Next, an example of the flow of the attribute information encoding process executed in step S104 in FIG. 11 will be described with reference to the flowchart in FIG.

[0149] When the attribute information encoding process is started, in step S111, the layering processing unit 111 of the attribute information encoding unit 104 performs layering processing to layer the attribute data and derive a difference value of the attribute data for each point. At this time, the layering processing unit 111 performs layering by applying the present technology described above in <1. Generation of Sub LoD>. Details of the layering process will be described later.

[0150] In step S112, the quantization unit 112 quantizes each of the difference values derived in step S111.

[0151] In step S113, the encoding unit 113 encodes the difference value quantized in step S112 to generate encoded data of the attribute data. If control information is generated in the layering process in step S111, the encoding unit 113 generates encoded data of the attribute data that also includes the control information.

[0152] When the process of step S113 ends, the attribute information encoding process ends, and the process returns to FIG.

[0153] By processing each step in this manner, the attribute information encoding unit 104 can generate a reference relationship of attribute data within a hierarchy so as to suppress bias in the prediction direction, thereby suppressing a decrease in encoding efficiency.

[0154] <Layering process flow> Next, an example of the flow of the layering process executed in step S111 of FIG. 12 will be described with reference to the flowchart of FIG.

[0155] When the layering process is started, in step S121, the control unit 121 of the layering processing unit 111 processes the attribute data of all points, and performs the processes of steps S122 to S126 to generate the first layer (LoD).

[0156] This first layer (the layer generated first) becomes the lowest layer in the layered attribute data by the process of step S127 described later. In other words, it can be said that the control unit 121 sets the processing target LoD to the lowest layer.

[0157] In step S122, the layering processing unit 122 sets reference points from among the points to be processed. A reference point is a point whose attribute data is referenced when predicting the attribute data of a prediction point. A prediction point is a point at which attribute data prediction is performed in this layer. In other words, it can be said that the layering processing unit 122 sets each point to be processed as either a prediction point or a reference point.

[0158] In step S123, the control unit 121 determines whether or not to generate a sub-layer (Sub LoD) in this layer. The control unit 121 determines whether or not to create a sub-layer in this layer based on any information, such as settings by a user or an application. If it is determined that a sub-layer should be created, the process proceeds to step S124.

[0159] In step S124, the Sub LoD generation unit 123 generates a sub-hierarchy within the hierarchy to be processed using the geometry data. That is, the Sub LoD generation unit 123 generates a reference relationship of attribute data within the hierarchy (LoD) as in "Method 1" in the top row of the table shown in Fig. 3 described above in <1. Generation of Sub LoD>.

[0160] Note that this generation method is arbitrary. For example, the Sub LoD generation unit 123 may apply "Method 1-1" in the second row from the top of the table shown in Fig. 3, or "Method 1-1-1" in the third row from the top of the table, or "Method 1-1-2" in the fourth row from the top of the table. Of course, the Sub LoD generation unit 123 may also apply other methods.

[0161] Once the sub-hierarchy is created, the process proceeds to step S125.

[0162] Also, if it is determined in step S123 that sub-layering is not to be performed on the layer to be processed, the process of step S124 is skipped and the process proceeds to step S125.

[0163] In step S125, the control unit 121 processes the attribute data of the reference point selected in step S122, and performs the processes of steps S122 to S126 to generate the next layer (LoD).

[0164] The next layer newly selected as the processing target becomes the layer immediately above the layer previously selected as the processing target in the layered attribute data by the processing in step S127 described later. In other words, it can be said that the control unit 121 updates the processing target LoD to the next higher layer.

[0165] In step S126, the control unit 121 determines whether all points have been processed. By repeating the above process, all points are set as prediction points (it is possible that prediction will not be performed on the last point). In other words, the control unit 121 determines whether all layers have been generated. If it is determined that there are points that have not yet been selected as prediction points and layering is not complete, the process returns to step S122.

[0166] That is, the processes of steps S122 to S126 are executed for the next layer that is newly selected as the processing target in step S125. That is, the point that was set as the previous reference point is set as either a prediction point or a reference point. In this way, the processes of steps S122 to S126 are recursively repeated for the point that was set as the reference point, each layer and each sub-layer are generated, and reference relationships between layers and sub-layers are generated.

[0167] By repeatedly executing the processes of steps S122 to S126, all hierarchical levels (and all sub-hierarchical levels) are generated, and if it is determined in step S126 that all points have been processed, the process proceeds to step S127.

[0168] In step S127, the inverting unit 124 inverts the hierarchy of the generated attribute data and assigns a hierarchy number to each hierarchy in the reverse order of the generation.

[0169] In step S128, the weighting unit 125 derives a weight value for the attribute data of each point, which reflects the reference relationship between the sub-hierarchies.

[0170] Furthermore, the weighting unit 125 predicts the attribute data for each point using the derived weight value and the reference relationship between the attribute data, and derives the difference value between the attribute data and the predicted value.

[0171] In step S129, the control unit 121 generates control information relating to the sub-layering of attribute data, and transmits it to the decoding side.

[0172] When the process of step S129 is completed, the layering process ends and the process returns to FIG.

[0173] By performing the processing at each step in this manner, the layering processing unit 111 can generate a reference relationship of attribute data within a layer so as to suppress bias in the prediction direction, and therefore the encoding device 100 can suppress a decrease in encoding efficiency.

[0174] 3. Second Embodiment <Decryption device> Next, another example of a device to which the present technology described above in <1. Generation of Sub LoD> is applied will be described. Fig. 14 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which the present technology is applied. The decoding device 200 shown in Fig. 14 is a device that decodes encoded data of a point cloud (3D data). The decoding device 200 decodes the encoded data of the point cloud by applying the present technology described above in <1. Generation of Sub LoD>.

[0175] Note that Fig. 14 shows the main processing units, data flows, etc., and does not necessarily show everything. That is, in the decoding device 200, there may be processing units that are not shown as blocks in Fig. 14, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 14.

[0176] As shown in FIG. 14, the decoding device 200 includes a decoding target LoD depth setting unit 201 , an encoded data extraction unit 202 , a position information decoding unit 203 , an attribute information decoding unit 204 , and a point cloud generation unit 205 .

[0177] The decoding target LoD depth setting unit 201 performs processing related to setting the depth of the layer (LoD) to be decoded. For example, the decoding target LoD depth setting unit 201 sets up to which layer to decode for the coded data of the point cloud held in the coded data extraction unit 202. The method of setting the layer depth to be decoded is arbitrary.

[0178] For example, the decoding target LoD depth setting unit 201 may set the depth based on an instruction related to the layer depth from an external source such as a user or an application. Alternatively, the decoding target LoD depth setting unit 201 may obtain and set the layer depth to be decoded based on any information such as an output image.

[0179] For example, the decoding target LoD depth setting unit 201 may set the hierarchical depth to be decoded based on the viewpoint position, direction, angle of view, viewpoint movement (movement, pan, tilt, zoom), etc. of the two-dimensional image generated from the point cloud.

[0180] The data unit for setting the layer depth to be decoded can be any. For example, the decoding target LoD depth setting unit 201 can set the layer depth for the entire point cloud, can set the layer depth for each object, or can set the layer depth for each partial region within an object. Of course, the layer depth can also be set in a data unit other than these examples.

[0181] The coded data extraction unit 202 acquires and holds the bitstream input to the decoding device 200. The coded data extraction unit 202 extracts coded data of geometry data (position information) and attribute data (attribute information) from the top layer up to the layer specified by the decoding target LoD depth setting unit 201 from the bitstream it holds. The coded data extraction unit 202 supplies the coded data of the extracted geometry data to the position information decoding unit 203. The coded data extraction unit 202 supplies the coded data of the extracted attribute data to the attribute information decoding unit 204.

[0182] The position information decoding unit 203 acquires the coded data of the geometry data supplied from the coded data extraction unit 202. The position information decoding unit 203 decodes the coded data of the geometry data to generate geometry data (decoded result). This decoding method may be any method as long as it is the same as that used by the position information decoding unit 102 of the encoding device 100. The position information decoding unit 203 supplies the generated geometry data (decoded result) to the attribute information decoding unit 204 and the point cloud generation unit 205.

[0183] The attribute information decoding unit 204 acquires the coded data of the attribute data supplied from the coded data extraction unit 202. The attribute information decoding unit 204 acquires the geometry data (decoded result) supplied from the position information decoding unit 203. The attribute information decoding unit 204 uses the position information (decoded result) to decode the coded data of the attribute data by the method to which the present technology is applied described above in <1. Generation of Sub LoD>, and generates attribute data (decoded result). The attribute information decoding unit 204 supplies the generated attribute data (decoded result) to the point cloud generation unit 205.

[0184] The point cloud generation unit 205 acquires geometry data (decoding result) supplied from the position information decoding unit 203. The point cloud generation unit 205 acquires attribute data (decoding result) supplied from the attribute information decoding unit 204. The point cloud generation unit 205 generates a point cloud (decoding result) using the geometry data (decoding result) and attribute data (decoding result). The point cloud generation unit 205 outputs the generated point cloud (decoding result) data to the outside of the decoding device 200.

[0185] With this configuration, the decoding device 200 can perform inverse layering by applying a plurality of layering techniques, and can therefore correctly inverse layering of attribute data layered by applying a plurality of layering techniques. That is, it is possible to correctly decode coded data that has been coded by applying such a plurality of layering techniques. This makes it possible to suppress a decrease in coding efficiency.

[0186] These processing units (the decoding target LoD depth setting unit 201 to the point cloud generation unit 205) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Furthermore, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using these. Of course, each processing unit may have both of these configurations, and may realize part of the above-mentioned processing by a logic circuit and other parts by executing a program. The configurations of the processing units may be independent of each other. For example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and by executing a program.

[0187] <Attribute information decoding unit> Fig. 15 is a block diagram showing an example of the main configuration of the attribute information decoding unit 204 (Fig. 14). Note that Fig. 15 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, the attribute information decoding unit 204 may include processing units not shown as blocks in Fig. 15, or processes and data flows not shown as arrows, etc. in Fig. 15.

[0188] As shown in FIG. 15, the attribute information decoding unit 204 includes a decoding unit 211, an inverse quantization unit 212, and an inverse layer processing unit 213.

[0189] The decoding unit 211 performs processing related to decoding of the coded data of the attribute data. For example, the decoding unit 211 obtains the coded data of the attribute data supplied to the attribute information decoding unit 204.

[0190] The decoding unit 211 decodes the coded data of the attribute data to generate attribute data (decoded result). This decoding method may be any method that corresponds to the coding method used by the coding unit 113 (FIG. 9) of the coding device 100. The generated attribute data (decoded result) corresponds to the attribute data before coding, and is a difference value between the attribute data and its predicted value, and is quantized, as described in the first embodiment. The decoding unit 211 supplies the generated attribute data (decoded result) to the inverse quantization unit 212.

[0191] If the coded data of the attribute data includes control information related to weight values and control information related to the layering of the attribute data, the decoding unit 211 also supplies this control information to the inverse quantization unit 212 .

[0192] The inverse quantization unit 212 performs processing related to inverse quantization of attribute data. For example, the inverse quantization unit 212 acquires attribute data (decoded result) supplied from the decoding unit 211. When control information is supplied from the decoding unit 211, the inverse quantization unit 212 also acquires the control information.

[0193] The inverse quantization unit 212 inverse quantizes the attribute data (decoded result). The inverse quantization unit 212 performs this inverse quantization using a method corresponding to the quantization performed by the quantization unit 112 (FIG. 9) of the encoding device 100. The inverse quantization unit 212 supplies the inverse quantized attribute data to the inverse layering processing unit 213. Furthermore, when control information is acquired from the decoding unit 211, the inverse quantization unit 212 also supplies the control information to the inverse layering processing unit 213.

[0194] The inverse layering processing unit 213 acquires the inversely quantized attribute data supplied from the inverse quantization unit 212. As described above, this attribute data is a difference value. The inverse layering processing unit 213 also acquires geometry data (decoded result) supplied from the position information decoding unit 203. The inverse layering processing unit 213 uses the geometry data to perform inverse layering on the acquired attribute data (difference value). This inverse layering is the reverse process of the layering performed by the layering processing unit 111 (FIG. 9) of the encoding device 100.

[0195] In this case, the de-hierarchizing processor 213 performs de-hierarchizing by applying the present technology described above in <1. Generation of Sub LoD>. For example, the de-hierarchizing processor 213 generates sub-hierarchies (Sub LoD) within the hierarchy of attribute data, generates reference relationships of attribute data between the sub-hierarchies, predicts attribute data using the reference relationships, and performs de-hierarchizing. The de-hierarchizing processor 213 supplies the de-hierarchized attribute data to the point cloud generator 205 ( FIG. 14 ) as the decoding result.

[0196] By performing the inverse hierarchical structure as described above, the attribute information decoding unit 204 can correctly inversely hierarchically structure attribute data that has been layered or sub-layered by applying the above-mentioned "Method 1" etc. Therefore, the attribute information decoding unit 204 can suppress a decrease in coding efficiency.

[0197] These processing units (the decoding unit 211 to the inverse layering processing unit 213) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Furthermore, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-described processing by executing a program using these. Of course, each processing unit may have both of these configurations, and may realize part of the above-described processing by a logic circuit and the other by executing a program. The configurations of the processing units may be independent of each other. For example, some processing units may realize part of the above-described processing by a logic circuit, other processing units may realize the above-described processing by executing a program, and still other processing units may realize the above-described processing by both a logic circuit and by executing a program.

[0198] <Reverse layering processing unit> Fig. 16 is a block diagram showing an example of the main configuration of the de-hierarchy processing unit 213 (Fig. 15). Note that Fig. 16 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 16. In other words, the de-hierarchy processing unit 213 may include processing units that are not shown as blocks in Fig. 16, and may include processing and data flows that are not shown as arrows, etc. in Fig. 16.

[0199] As shown in FIG. 16, the de-layering processor 213 includes a control unit 221, a layering processor 222, a Sub LoD generator 223, an inverter 224, and a de-layering processor 225.

[0200] The control unit 221 performs processing related to layering control. For example, the control unit 221 acquires inversely quantized attribute data supplied from the inverse quantization unit 212 ( FIG. 15 ). The control unit 221 also acquires geometry data (decoded results) supplied from the position information decoding unit 203. The control unit 221 supplies the acquired attribute data and geometry data to the layering processing unit 222. Furthermore, when control information is supplied from the inverse quantization unit 212, the control unit 221 also acquires the control information and supplies it to the layering processing unit 222.

[0201] The control unit 221 also controls the layering processing unit 222 and the Sub LoD generation unit 223 to perform layering and sub-layering on the attribute data, and to generate layers and sub-layers with structures similar to those of the layers and sub-layers generated during encoding. For example, the control unit 221 applies the present technology described above in <1. Generation of Sub LoD> to perform layering and sub-layering. For example, the control unit 221 causes sub-layering to be performed on desired layers. That is, the control unit 221 can cause sub-layering to be performed on some layers or on all layers.

[0202] For example, the control unit 221 may perform sub-layering based on control information regarding sub-layering of attribute data, as in "Method 2-2" in the tenth row from the top of the table shown in FIG.

[0203] For example, the control unit 221 may cause the attribute data to be sub-layered based on control information (for example, sub_lod_enable_flag) indicating whether or not sub-layering of the attribute data is permitted.

[0204] Furthermore, the control unit 221 may perform sub-layering (sampling) of attribute data based on control information (for example, sub_lod_distance) indicating the interval at which attribute data is sampled.

[0205] Furthermore, the control unit 221 may perform sub-hierarchical layering (sampling) of attribute data based on control information (for example, sub_lod_mode) indicating a sampling method for attribute data.

[0206] By using such control information transmitted from the encoding side, the control unit 221 can more easily perform sub-layering similar to that on the encoding side.

[0207] The layering processing unit 222 performs processing related to layering of attribute data. For example, the layering processing unit 222 acquires attribute data and geometry data (decoding results) supplied from the control unit 221.

[0208] The layering processing unit 222 layers the acquired attribute data using the acquired geometry data under the control of the control unit 221. This layering method may be the same as the layering by the layering processing unit 122. That is, the layering processing unit 222 layers the attribute data in the same structure as that generated by the layering processing unit 122. In other words, the hierarchical structure of the attribute data generated by the layering processing unit 122 is reproduced. The layering processing unit 222 supplies the layered attribute data and geometry data to the Sub LoD generation unit 223.

[0209] The Sub LoD generation unit 223 performs processing related to sub-layering. For example, the Sub LoD generation unit 223 acquires layered attribute data and geometry data supplied from the layering processing unit 222.

[0210] The Sub LoD generating unit 223 generates a sub-layer (Sub LoD) within the layer of the attribute data under the control of the control unit 221. The Sub LoD generating unit 223 performs such sub-layering using geometry data.

[0211] In other words, the Sub LoD generation unit 223 generates sub-hierarchies (Sub LoDs) of attribute data within a hierarchy (LoD) and generates reference relationships between those sub-hierarchies, as in "Method 2-1" in the ninth row from the top of the table shown in Figure 3 described above in <1. Generation of Sub LoD>.

[0212] In this way, the Sub LoD generating unit 223 can generate a reference relationship of attribute data within a layer so as to suppress bias in the prediction direction, and can suppress a decrease in coding efficiency.

[0213] The Sub LoD generation unit 223 performs this sub-hierarchy generation using the same method as the sub-hierarchy generation by the Sub LoD generation unit 123. That is, the Sub LoD generation unit 223 generates a sub-hierarchy with the same structure as that generated by the Sub LoD generation unit 123. In other words, the sub-hierarchy structure of the attribute data generated by the Sub LoD generation unit 123 is reproduced.

[0214] For example, the Sub LoD generation unit 223 may recursively repeat the process of sampling (selecting) a portion of the attribute data for each point in a hierarchy to create a sub-hierarchy, as in "Method 1-1" in the second row from the top of the table shown in Figure 3.

[0215] Alternatively, the Sub LoD generation unit 223 may generate a sub-hierarchy by arranging the attribute data of the hierarchy to be processed in Morton order and sampling a portion of the data at equal intervals, as in "Method 1-1-1" in the third row from the top of the table shown in Figure 3.

[0216] In addition, the Sub LoD generation unit 223 may generate a sub-hierarchy by arranging the attribute data of the hierarchy to be processed in Morton order and sampling some of it at non-uniform intervals, as in "Method 1-1-2" in the fourth row from the top of the table shown in Figure 3.

[0217] Of course, other methods may be used. Note that if sub-layering is not permitted by the control unit 221, the Sub LoD generation unit 223 can omit sub-layering. In other words, the Sub LoD generation unit 223 can perform sub-layering only on layers permitted by the control unit 221.

[0218] The Sub LoD generating unit 223 supplies the attribute data thus sub-hierarchized under the control of the control unit 221 to the inverting unit 224 .

[0219] The inversion unit 224 performs processing related to layer inversion. For example, the inversion unit 224 acquires attribute data supplied from the Sub LoD generation unit 223.

[0220] The inversion unit 224 inverts the hierarchy of the attribute data in the same manner as the inversion unit 124. For example, the inversion unit 224 assigns a hierarchy number (a number for identifying a hierarchy in which the highest hierarchy is 0, the value is incremented by 1 for each hierarchy level, and the lowest hierarchy is the maximum value) to each hierarchy level of the attribute data in the reverse order of the generation of the attribute data.

[0221] The inversion unit 224 supplies the attribute data with the inverted hierarchy to the inverse hierarchy processing unit 225 .

[0222] The inverse-hierarchy processing unit 225 performs processing related to inverse hierarchical construction. For example, the inverse-hierarchy processing unit 225 acquires attribute data supplied from the inverter 224. This attribute data is composed of difference values between the attribute data and its predicted values, and has a hierarchical structure generated (reproduced) by the hierarchical construction processing unit 222 and a sub-hierarchical structure generated (reproduced) by the Sub LoD generation unit 223. The inverse-hierarchy processing unit 225 inversely processes this acquired attribute data, and generates (restores) the attribute data for each point.

[0223] In other words, the inverse-hierarchy processing unit 225 predicts the attribute data of each point and derives (restores) the attribute data of each point from the difference value using the derived predicted value. In doing so, the inverse-hierarchy processing unit 225 predicts the attribute data (derives the predicted value of the attribute data) using the reference relationships between hierarchies and sub-hierarchies reproduced as described above, as in "Method 2" in the eighth row from the top of the table shown in Fig. 3.

[0224] For example, the inverse layering processing unit 225 derives predicted values of the attribute data of each point for each layer in order from the highest layer to the lowest layer (within a layer, for each sub-layer in order from the highest sub-layer to the lowest sub-layer).

[0225] The inverse-hierarchy processing unit 225 then derives the attribute data for that point by adding the derived predicted value to the difference value corresponding to that point. In this way, the inverse-hierarchy processing unit 225 generates (restores) the attribute data for each point.

[0226] The inverse layer processing unit 225 supplies the attribute data (decoded result) generated (restored) by the inverse layer processing described above to the point cloud generating unit 205 (FIG. 14).

[0227] By performing the inverse layering as described above, the inverse layering processor 225 can correctly inverse layering the attribute data that has been layered or sub-layered using the above-mentioned "Method 1" etc. Therefore, the decoding device 200 can suppress a decrease in encoding efficiency.

[0228] These processing units (control unit 221 to reverse layering processing unit 225) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Furthermore, each processing unit may have, for example, a CPU, ROM, RAM, etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may realize some of the above-described processing using a logic circuit and other processing by executing a program. The configurations of the processing units may be independent of each other. For example, some processing units may realize some of the above-described processing using a logic circuit, other processing units may execute a program to realize the above-described processing, and still other processing units may realize the above-described processing using both a logic circuit and by executing a program.

[0229] <Decryption process flow> Next, a description will be given of the processing executed by the decoding device 200. The decoding device 200 decodes the encoded data of the point cloud by executing a decoding process. An example of the flow of this decoding process will be described with reference to the flowchart in FIG.

[0230] When the decoding process is started, the decoding target LoD depth setting unit 201 of the decoding device 200 sets the LoD depth to be decoded (that is, the range of layers to be decoded) in step S201.

[0231] In step S202, the coded data extraction unit 202 acquires and holds the bitstream, and extracts coded data of geometry data (position information) and attribute data (attribute information) from the top layer to the layer of the LoD depth set in step S201. The layer of the extracted geometry data and the layer (number of layers) of the attribute data may or may not match each other.

[0232] In step S203, the position information decoding unit 203 decodes the coded data of the geometry data extracted in step S202 to generate geometry data (decoded result).

[0233] In step S204, the attribute information decoding unit 204 decodes the coded data of the attribute data extracted in step S202 to generate attribute data (decoded result). At this time, the attribute information decoding unit 204 performs processing by applying the present technology described above in <1. Generation of Sub LoD>. Details of the attribute information decoding processing will be described later.

[0234] In step S205, the point cloud generation unit 205 generates and outputs point cloud data (decoding result) using the geometry data (decoding result) generated in step S203 and the attribute data (decoding result) generated in step S204.

[0235] When the process of step S205 ends, the decoding process ends.

[0236] By performing the processing of each step in this manner, the decoding device 200 can correctly de-layer the attribute data that has been layered or sub-layered by applying the above-mentioned "Method 1" etc. Therefore, the decoding device 200 can suppress a decrease in encoding efficiency.

[0237] <Flow of attribute information decoding process> Next, an example of the flow of the attribute information decoding process executed in step S204 of FIG. 17 will be described with reference to the flowchart of FIG.

[0238] When the attribute information decoding process starts, in step S211, the decoding unit 211 of the attribute information decoding unit 204 decodes the coded data of the attribute data to generate attribute data (decoded result). This attribute data (decoded result) has been quantized as described above.

[0239] In step S212, the inverse quantization unit 212 performs inverse quantization processing to inversely quantize the attribute data (decoding result) generated in step S211. This inversely quantized attribute data is a difference value.

[0240] In step S213, the inverse layering processor 213 performs inverse layering processing to inverse layer the attribute data inversely quantized in step S212 and derive attribute data for each point. At this time, the inverse layering processor 213 performs inverse layering by applying the present technology described above in <1. Generation of Sub LoD>. Details of the inverse layering processing will be described later.

[0241] When the process of step S213 ends, the attribute information decoding process ends, and the process returns to FIG.

[0242] By performing the processing of each step in this manner, the attribute information decoding unit 204 can correctly de-hierarchize the attribute data that has been hierarchized and sub-hierarchized by applying the above-mentioned "Method 1" etc. Therefore, the decoding device 200 can suppress a decrease in encoding efficiency.

[0243] <Flow of reverse layering process> Next, an example of the flow of the layer inversion process executed in step S213 of FIG. 18 will be described with reference to the flowchart of FIG.

[0244] When the inverse layering process is started, in step S221, the control unit 221 of the inverse layering processing unit 213 processes the attribute data (difference values) of all points, as in step S121 (FIG. 13), and performs the processes of steps S222 to S226 to generate the first layer (LoD).

[0245] This first layer (the layer generated first) becomes the lowest layer in the layered attribute data by the process of step S227 described later. In other words, it can be said that the control unit 221 sets the processing target LoD to the lowest layer.

[0246] In step S222, the layering processing unit 222 sets reference points from among the points to be processed, similar to step S122 (FIG. 13). In other words, it can be said that the layering processing unit 222 sets each point to be processed as either a prediction point or a reference point.

[0247] In step S223, the control unit 221 determines whether or not to generate a sub-layer (Sub LoD) in this layer, similar to step S123 (FIG. 13). The control unit 221 can make this determination based on any information, etc. For example, the control unit 221 determines whether or not to create a sub-layer in this layer based on control information, etc. transmitted from the encoding side. If it is determined that a sub-layer should be created, the process proceeds to step S224.

[0248] In step S224, the Sub LoD generation unit 223 generates a sub-hierarchy within the hierarchy to be processed using geometry data, similar to step S124 (FIG. 13). That is, the Sub LoD generation unit 223 generates a sub-hierarchy (Sub LoD) of attribute data within the hierarchy (LoD) and generates reference relationships between the sub-hierarchies, as in "Method 2-1" in the ninth row from the top of the table shown in FIG. 3 described above in <1. Generation of Sub LoD>.

[0249] That is, in step S224, the Sub LoD generation unit 223 performs this sub-hierarchy generation using the same method as the sub-hierarchy generation performed in step S124. That is, the Sub LoD generation unit 223 generates a sub-hierarchy with the same structure as that generated in step S124. In other words, the sub-hierarchy structure of the attribute data generated in step S124 is reproduced.

[0250] For example, the Sub LoD generation unit 223 may apply "Method 1-1" in the second row from the top of the table shown in Fig. 3, or "Method 1-1-1" in the third row from the top of the table, or "Method 1-1-2" in the fourth row from the top of the table. Of course, the Sub LoD generation unit 223 may apply other methods.

[0251] Once the sub-hierarchy is created, the process proceeds to step S225.

[0252] Also, if it is determined in step S223 that sub-layering is not to be performed for the layer to be processed, the process of step S224 is skipped and the process proceeds to step S225.

[0253] In step S225, the control unit 221 processes the attribute data of the reference point selected in step S222, as in step S125 (Figure 13), and performs the processes of steps S222 to S226 to generate the next layer (LoD).

[0254] The next layer newly selected as the processing target becomes the layer immediately above the layer previously selected as the processing target in the layered attribute data by the processing in step S227 described later. In other words, it can be said that the control unit 221 updates the processing target LoD to the next higher layer.

[0255] In step S226, the control unit 221 determines whether all points have been processed, similarly to step S126 (FIG. 13). By repeating the above process, all points are set as prediction points (it is possible that prediction is not performed for the last point). In other words, the control unit 221 determines whether all layers have been generated. If it is determined that there are points that have not yet been selected as prediction points and layering is not complete, the process returns to step S222.

[0256] That is, the processes of steps S222 to S226 are executed for the next layer that is newly selected as the processing target in step S225. That is, the point that was set as the previous reference point is set as either a prediction point or a reference point. In this way, the processes of steps S222 to S226 are recursively repeated for the point that was set as the reference point, each layer and each sub-layer are generated, and reference relationships between layers and sub-layers are generated (reproduced).

[0257] By repeatedly executing the processes of steps S222 to S226, all layers (and all sub-layers) are generated (reproduced), and if it is determined in step S226 that all points have been processed, the process proceeds to step S227.

[0258] In step S227, the inverting unit 224 inverts the hierarchy of the generated attribute data, as in step S127 (FIG. 13), and assigns hierarchy numbers to each hierarchy in the reverse order of generation. As a result, attribute data is generated (reproduced), which is made up of difference values between attribute data and their predicted values, and has the hierarchical structure generated (reproduced) in step S222 and the sub-hierarchical structure generated (reproduced) in step S224.

[0259] In step S228, the inverse layer processing unit 225 inversely layers the generated attribute data, and generates (restores) attribute data for each point.

[0260] In other words, the inverse-hierarchy processing unit 225 predicts the attribute data of each point and derives (restores) the attribute data of each point from the difference value using the derived predicted value. In doing so, the inverse-hierarchy processing unit 225 predicts the attribute data (derives the predicted value of the attribute data) using the reference relationships between hierarchies and sub-hierarchies reproduced as described above, as in "Method 2" in the eighth row from the top of the table shown in Fig. 3.

[0261] For example, the inverse layering processing unit 225 derives predicted values of the attribute data of each point for each layer in order from the highest layer to the lowest layer (within a layer, for each sub-layer in order from the highest sub-layer to the lowest sub-layer).

[0262] The inverse-hierarchy processing unit 225 then derives the attribute data for that point by adding the derived predicted value to the difference value corresponding to that point. In this way, the inverse-hierarchy processing unit 225 generates (restores) the attribute data for each point.

[0263] When attribute data for all points has been generated (restored) and the inverse layering is complete, the inverse layering process ends and the process returns to FIG.

[0264] By performing the processing of each step in this manner, the de-layering processor 213 can correctly de-layer the attribute data that has been layered or sub-layered by applying the above-mentioned "Method 1" etc. Therefore, the decoding device 200 can suppress a decrease in encoding efficiency.

[0265] <4. Notes> <Hierarchization / reverse hierarchy method> Although Lifting has been used as an example of a method for layering and delayering attribute information in the above description, this technology can be applied to any technology for layering attribute information. In other words, the method for layering and delayering attribute information may be other than Lifting.

[0266] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0267] FIG. 20 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0268] In a computer 900 shown in FIG. 20, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0269] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0270] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0271] In a computer configured as above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0272] The program executed by the computer can be applied by recording it on removable media 921 such as package media, for example. In this case, the program can be installed in storage unit 913 via input / output interface 910 by inserting removable media 921 into drive 915.

[0273] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0274] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0275] <Applicable targets of this technology> While the above describes the application of this technology to the encoding and decoding of point cloud data, this technology is not limited to these examples and can be applied to the encoding and decoding of 3D data of any standard. In other words, as long as it does not conflict with the above-described technology, various processes such as encoding and decoding methods and specifications of various data such as 3D data and metadata are arbitrary. Furthermore, as long as it does not conflict with the above-described technology, some of the above-described processes and specifications may be omitted.

[0276] Furthermore, although the encoding device 100 and the decoding device 200 have been described above as application examples of the present technology, the present technology can be applied to any configuration.

[0277] For example, this technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) used in satellite broadcasting, cable TV and other wired broadcasting, distribution over the Internet, and distribution to terminals via cellular communications, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, or play images from these storage media.

[0278] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0279] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, AV (Audio Visual) equipment, a portable information processing terminal, or an IoT (Internet of Things) device.

[0280] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0281] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications thereof are also arbitrary.

[0282] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be assumed not only to include the identification information in the bit stream, but also to include difference information of the identification information relative to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information relative to the reference information.

[0283] Furthermore, various types of information (metadata, etc.) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, making one piece of data available (linked) when processing the other piece of data. In other words, mutually associated data may be combined into one piece of data or may be individual pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of the same recording medium). Note that this "association" may refer to only a portion of the data, rather than the entire data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0284] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0285] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0286] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0287] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0288] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0289] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0290] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0291] The present technology can also be configured as follows. (1) a layering unit that layers attribute information of a point cloud that represents a three-dimensional object as a set of points and generates a reference relationship between the attribute information between layers; a sub-hierarchy generation unit that generates sub-hierarchies of the attribute information in the hierarchy of the attribute information generated by the hierarchy generation unit and generates reference relationships of the attribute information between the sub-hierarchies; An information processing device comprising: (2) The sub-hierarchy generation unit generates sub-hierarchies of the attribute information by arranging the attribute information of the processing target hierarchy in Morton order and sampling the attribute information at equal intervals. The information processing device described in (1). (3) The sub-hierarchy generation unit generates sub-hierarchies of the attribute information by arranging the attribute information of the processing target hierarchy in Morton order and sampling the attribute information at non-equidistant intervals. The information processing device described in (1). (4) a weighting unit that derives a weight value for each of the attribute information by updating the weight value for each of the layers in accordance with the reference relationship of the attribute information between the sub-layers generated by the sub-layering unit; The information processing device according to any one of (1) to (3), further comprising: (5) a weighting unit that derives a weight value for each of the attribute information items according to the reference relationship of the attribute information items between the layers generated by the layering unit and the reference relationship of the attribute information items between the sub-layers generated by the sub-layering unit; The information processing device according to any one of (1) to (3), further comprising: (6) an encoding unit that encodes the attribute information and generates encoded data of the attribute information; The information processing device according to any one of (1) to (5), further comprising: (7) A generating unit that generates control information related to sub-hierarchical division of the attribute information, The encoding unit generates the encoded data including the control information generated by the generation unit. (6) An information processing device according to (6). (8) The generating unit generates control information indicating whether sub-hierarchy of the attribute information is permitted. (7) An information processing device according to (7). (9) The generating unit generates control information indicating an interval at which the attribute information is sampled by the sub-layering unit. An information processing device according to (7) or (8). (10) The generating unit generates control information indicating a method for sampling the attribute information by the sub-layering unit. An information processing device according to any one of (7) to (9). (11) Layering attribute information of a point cloud that represents a three-dimensional object as a set of points, and generating a reference relationship between the attribute information layers; In the generated hierarchy of the attribute information, the attribute information of the hierarchy is divided into sub-hierarchies, and a reference relationship of the attribute information between the sub-hierarchies is generated. Information processing methods.

[0292] (12) a layering unit that layers attribute information of a point cloud that represents a three-dimensional object as a set of points and generates a reference relationship between the attribute information between layers; a sub-hierarchy generation unit that generates sub-hierarchies of the attribute information in the hierarchy of the attribute information generated by the hierarchy generation unit and generates reference relationships of the attribute information between the sub-hierarchies; a reverse-hierarchy generation unit that reverses the hierarchy of the attribute information based on the reference relationship of the attribute information between the hierarchies generated by the hierarchy generation unit and the reference relationship of the attribute information between the sub-hierarchies generated by the sub-hierarchy generation unit; An information processing device comprising: (13) The sub-hierarchy generation unit generates sub-hierarchies of the attribute information by arranging the attribute information of the processing target hierarchy in Morton order and sampling the attribute information at equal intervals. (12) An information processing device according to (12). (14) The sub-hierarchy generation unit generates sub-hierarchies of the attribute information by arranging the attribute information of the processing target hierarchy in Morton order and sampling the attribute information at non-equidistant intervals. (12) An information processing device according to (12). (15) A decoding unit that decodes the coded data of the attribute information and restores the attribute information. The information processing device according to any one of (12) to (14), further comprising: (16) The decoding unit decodes the encoded data to restore control information related to sub-hierarchical division of the attribute information, The sub-layering unit sub-layers the attribute information based on the control information restored by the decoding unit. (15) An information processing device according to (15). (17) The sub-layering unit sub-layers the attribute information based on control information indicating whether sub-layering of the attribute information restored by the decoding unit is permitted. (16) An information processing device according to (16). (18) The sub-layering unit sub-layers the attribute information based on control information indicating a sampling interval of the attribute information restored by the decoding unit. The information processing device according to (16) or (17). (19) The sub-layering unit sub-layers the attribute information based on control information indicating a sampling method of the attribute information restored by the decoding unit. An information processing device according to any one of (16) to (18). (20) Layering attribute information of a point cloud that represents a three-dimensional object as a set of points, and generating a reference relationship between the attribute information layers; In the generated hierarchy of the attribute information, the attribute information of the hierarchy is divided into sub-hierarchies, and a reference relationship of the attribute information between the sub-hierarchies is generated; The attribute information is reversed in hierarchy based on the reference relationships of the attribute information between the generated hierarchies and the reference relationships of the attribute information between the generated sub-hierarchies. Information processing methods. [Explanation of symbols]

[0293] 100 Encoding device, 101 Position information encoding unit, 102 Position information decoding unit, 103 Point cloud generation unit, 104 Attribute information encoding unit, 105 Bitstream generation unit, 111 Layering processing unit, 112 Quantization unit, 113 Encoding unit, 121 Control unit, 122 Layering processing unit, 123 Sub-LoD generation unit, 124 Inversion unit, 125 Weighting unit, 200 Decoding device, 201 Decoding target LOD depth setting unit, 202 Encoded data extraction unit, 203 Position information decoding unit, 204 Attribute information decoding unit, 205 Point cloud generation unit, 211 Decoding unit, 212 Inverse quantization unit, 213 Inverse layering processing unit, 221 Control unit, 222 Layering processing unit, 223 Sub-LoD generation unit, 224 Inversion section, 225 Inverse layer processing section< / lifting> < / octree>

Claims

1. a decoding unit that decodes coded data of a point cloud that represents a three-dimensional object as a set of points to obtain geometry data and attribute data; a reverse-hierarchy processing unit that reverses the hierarchy of the attribute data using the geometry data, generates sub-hierarchies within the hierarchy of the attribute data, and generates reference relationships of the attribute data between the sub-hierarchies; a weighting unit that reflects the reference relationship between the sub-hierarchies in a weight value of the attribute data for each of the hierarchies; A decoding processing device comprising:

2. The reference relationship represents the distance or number of references between the sub-hierarchies. The decoding processing device according to claim 1 .

3. The reference relationship represents the number of references, The weighting unit multiplies the weight value by a coefficient according to the number of references. The decoding processing device according to claim 2 .

4. the sub-hierarchy includes a first hierarchy and a second hierarchy; The weighting unit updates the weight values of the second layer in response to the update of the weight values of the first layer. The decoding processing device according to claim 1 .

5. The weighting unit multiplies the weight value of the first layer by a first coefficient α and multiplies the weight value of the second layer by a second coefficient (1-α). The decoding processing device according to claim 4 .

6. The decoding unit obtains control information indicating whether sub-hierarchy of the attribute data is permitted, The inverse layer generation processing unit generates the sub-layer based on the control information. The decoding processing device according to claim 1 .

7. The attribute data to be inverted into a layer is layered data to which a lifting process, which is a transformation process of attributes using the geometry data in encoding, has been applied. The decoding processing device according to claim 1 .

8. A decoding processing device, Decoding encoded data of a point cloud that represents a three-dimensional object as a set of points to obtain geometry data and attribute data; Inverting the hierarchy of the attribute data using the geometry data; generating a sub-hierarchy within the hierarchy of the attribute data; generating a reference relationship of the attribute data between the sub-hierarchies; The reference relationships between the sub-hierarchies are reflected in weight values of the attribute data for each of the hierarchies. Decryption process method.

9. The reference relationship represents the distance or number of references between the sub-hierarchies. The decoding method according to claim 8.

10. The reference relationship represents the reference number, Reflecting the weight value includes multiplying the weight value by a coefficient according to the number of references. The decoding method according to claim 9.

11. The sub-hierarchy includes a first hierarchy and a second hierarchy; The multiplication of the weight values includes updating the weight values of the second layer in response to updating the weight values of the first layer. The decoding method according to claim 8.

12. The multiplication of the weight values includes multiplying the weight values of the first hierarchy by a first coefficient α and multiplying the weight values of the second hierarchy by a second coefficient (1-α). The decoding method according to claim 11.

13. The decoding of the encoded data includes obtaining control information indicating whether sub-layering of the attribute data is permitted; The sub-layer is generated based on the control information. The decoding method according to claim 8.

14. The attribute data to be inverted into a layer is layered data to which a lifting process, which is a transformation process of attributes using the geometry data in encoding, has been applied. The decoding method according to claim 8.

Images