Information processing device and method

Abstract

The present disclosure pertains to an information processing device and method by which scalable decoding of point cloud data can be more easily achieved. In encoding of a point cloud that represents a three-dimensionally shaped object as a set of points, the present invention carries out control so as to prohibit the combined use of scalable encoding, which is an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, which is an encoding method associated with a change in the tree structure of geometric data. The present disclosure can be applied to, for example, an information processing device, an image processing device, an encoding device, a decoding device, an electronic apparatus, an information processing method, or a program.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that enable scalable decoding of point cloud data to be more easily achieved. [Background technology]

[0002] Conventionally, a method for encoding 3D data representing a three-dimensional structure such as a point cloud has been considered (see, for example, Non-Patent Document 1). Also, a method for encoding the encoded data of this point cloud has been proposed that enables scalable decoding (see, for example, Non-Patent Document 2). In the case of the method described in Non-Patent Document 2, scalable decoding is realized by making the reference structure of the attribute data similar to the tree structure of the geometry data.

[0003] Incidentally, a method has been proposed for scaling geometry data and thinning out points when encoding such a point cloud (see, for example, Non-Patent Document 3). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] R. Mekuria, Student Member IEEE, K. Blom, P. Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video",tcsvt_paper_submitted_february.pdf [Non-Patent Document 2] Ohji Nakagami, Satoru Kuma, "[G-PCC] Spatial scalability support for G-PCC", ISO / IEC JTC1 / SC29 / WG11 MPEG2019 / m47352, March 2019, Geneva, CH [Non-Patent Document 3] Xiang Zhang, Wen Gao, Sehoon Yea, Shan Liu, "[G-PCC][New proposal] Signaling delta QPs for adaptive geometry quantization in point cloud coding", ISO / IEC JTC1 / SC29 / WG11 MPEG2019 / m49232, July 2019 Gothenburg, Sweden Summary of the Invention [Problem to be solved by the invention]

[0005] However, as described in Non-Patent Document 3, when the geometry data is scaled and points are thinned, the tree structure of the geometry data changes. Therefore, there is a risk that a mismatch occurs between the reference structure of the attribute data and the tree structure of the geometry data, making it impossible to perform scalable decoding. In other words, in order to realize scalable decoding when scaling the geometry data as described in Non-Patent Document 3, it is necessary to form the reference structure of the attribute data in accordance with the scaling of the geometry data. In other words, in order to encode point cloud data so that scalable decoding is possible, the attribute data reference structure must be changed depending on whether or not the geometry data is scaled, and there is a risk that complicated processing is required.

[0006] The present disclosure has been made in light of these circumstances, and makes it possible to more easily realize scalable decoding of point cloud data. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present technology is an information processing device that includes an encoding control unit that performs control to prohibit the combined use of scalable encoding, which is an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, which is an encoding method that involves changing the tree structure of geometry data, in encoding a point cloud that represents a three-dimensional object as a collection of points.

[0008] An information processing method according to one aspect of the present technology includes: An information processing device, This is an information processing method that, in encoding a point cloud that represents a three-dimensional object as a collection of points, controls to prohibit the combined use of scalable encoding, which is an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, which is an encoding method that involves changing the tree structure of geometry data.

[0009] In an information processing device and method according to one aspect of the present technology, in encoding a point cloud that represents a three-dimensional object as a collection of points, control is performed to prohibit the use of both scalable encoding, which is an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, which is an encoding method that involves modifying the tree structure of the geometry data. [Brief description of the drawings]

[0010] [Figure 1] FIG. 10 is a diagram illustrating an example of hierarchical structure of geometry data. [Diagram 2] FIG. 13 is a diagram illustrating an example of lifting. [Diagram 3] FIG. 13 is a diagram illustrating an example of lifting. [Figure 4] FIG. 13 is a diagram illustrating an example of lifting. [Diagram 5] FIG. 13 is a diagram illustrating an example of quantization. [Figure 6] FIG. 10 is a diagram illustrating an example of hierarchical organization of attribute data. [Figure 7] FIG. 13 is a diagram illustrating an example of inverse hierarchical organization of attribute data. [Figure 8] FIG. 13 is a diagram illustrating an example of geometry scaling. [Figure 9] FIG. 13 is a diagram illustrating an example of geometry scaling. [Figure 10] FIG. 10 is a diagram illustrating an example of hierarchical organization of attribute data. [Figure 11] FIG. 11 is a diagram illustrating an example of coding control. [Figure 12] FIG. 13 is a diagram for explaining an example of semantics. [Figure 13] FIG. 11 is a diagram illustrating an example of a profile. [Figure 14] FIG. 11 is a diagram for explaining an example of syntax. [Figure 15] FIG. 2 is a block diagram showing an example of the main configuration of an encoding device. [Figure 16] 11 is a block diagram showing an example of the main configuration of a geometry data encoding unit. FIG. [Figure 17] 11 is a block diagram showing an example of the main configuration of an attribute data encoding unit. [Figure 18] 11 is a flowchart illustrating an example of the flow of an encoding control process. [Figure 19] 11 is a flowchart illustrating an example of the flow of an encoding process. [Figure 20] 13 is a flowchart illustrating an example of the flow of a geometry data encoding process. [Figure 21] 13 is a flowchart illustrating an example of the flow of an attribute data encoding process. [Figure 22] FIG. 2 is a block diagram showing an example of the main configuration of a decoding device. [Diagram 23] 13 is a flowchart illustrating an example of the flow of a decoding process. [Figure 24] FIG. 1 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Encoding Control 2. First embodiment (encoding device) 3. Second embodiment (decoding device) 4. Notes

[0012] <1. Encoding control> <References supporting technical content and technical terms> The scope of disclosure of the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent documents that were publicly known at the time of filing the application.

[0013] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: Khaled Mammou, Alexis Tourapis, Jungsun Kim, Fabrice Robinet, Valery Valentin, Yeping Su, "Lifting Scheme for Lossy Attribute Encoding in TMC1", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42640, April 2018, San Diego, US

[0014] In other words, the contents described in the above-mentioned non-patent documents and the contents of other documents referenced in the above-mentioned non-patent documents are also used as the basis for determining the support requirements.

[0015] <Point cloud> Previously, there was 3D data such as point clouds, which represent three-dimensional structures using point position information and attribute information, and meshes, which are composed of vertices, edges, and faces and define three-dimensional shapes using polygonal representations.

[0016] For example, in the case of a point cloud, a three-dimensional structure (an object having a three-dimensional shape) is represented by a large number of points. Point cloud data (also referred to as point cloud data) is composed of geometry data (also referred to as position information) and attribute data (also referred to as attribute information) of each point. The attribute data can include any information. For example, the attribute data may include color information, reflectance information, normal information, etc. of each point. In this way, point cloud data has a relatively simple data structure, and by using a sufficiently large number of points, any three-dimensional structure can be represented with sufficient accuracy.

[0017] <Quantization of position information using voxels> Since such point cloud data has a relatively large amount of data, an encoding method using voxels was devised to compress the amount of data by encoding, etc. A voxel is a three-dimensional region for quantizing geometry data (position information).

[0018] That is, a three-dimensional region (also called a bounding box) containing a point cloud is divided into small three-dimensional regions called voxels, and each voxel indicates whether it contains a point or not. In this way, the position of each point is quantized in voxel units. Therefore, by converting point cloud data into such voxel data (also called voxel data), it is possible to suppress an increase in the amount of information (typically, to reduce the amount of information).

[0019] For example, as shown in A of FIG. 1, it is assumed that a bounding box 10 is divided into a plurality of voxels 11-1 represented by small rectangles. For the sake of simplicity, the three-dimensional space is described as a two-dimensional plane. In other words, in reality, the bounding box 10 is a three-dimensional spatial region, and the voxels 11-1 are small regions of rectangular parallelepipeds (including cubes). The geometry data (position information) of each point 12-1 of the point cloud data represented by a black circle in A of FIG. 1 is corrected so that it is positioned for each voxel 11-1. In other words, the geometry data is quantized with the voxel as a unit. Note that all the squares in the bounding box 10 in A of FIG. 1 are voxels 11-1, and all the black circles shown in A of FIG. 1 are points 12-1.

[0020] <Tree structure (Octree)> Furthermore, a method was devised to make it possible to decode geometry data in a scalable manner by forming the geometry data into a tree structure. In other words, by making it possible to decode nodes from the top layer of the tree structure to any layer, it becomes possible to restore not only the geometry data at the highest resolution (lowest layer), but also at lower resolutions (intermediate layers). In other words, it is possible to decode at any resolution without decoding information at unnecessary layers (resolutions).

[0021] This tree structure may be of any type. For example, it may be a KD tree or an octree. An octree is an octet tree, and is suitable for dividing a three-dimensional space region (dividing it into two in each of the x, y, and z directions). In other words, as described above, it is suitable for a structure that divides the bounding box 10 into multiple voxels.

[0022] For example, one voxel is divided into two in each of the x, y, and z directions (i.e., divided into eight) to form a voxel in the next lower layer (also called LoD). In other words, two voxels arranged in each of the x, y, and z directions (i.e., eight voxels) are integrated to form a voxel in the next higher layer (LoD). By recursively repeating such a structure, an octree can be constructed using voxels.

[0023] In addition, in the voxel data, it is indicated whether each voxel contains a point or not. In other words, in the voxel data, the position of the point is expressed with a resolution of the voxel size. Therefore, by constructing an octree using the voxel data, it is possible to realize scalability of the resolution of the geometry data. In other words, it is easier to construct an octree of the geometry data by structuring the voxel data into a tree than by structuring points scattered at arbitrary positions into a tree.

[0024] For example, in the case of A in FIG. 1, since it is two-dimensional, four voxels 11-1 arranged vertically and horizontally as in B in FIG. 1 are integrated to form a voxel 11-2 on the next level, indicated by a thick line. Then, the geometry data is quantized using this voxel 11-2. That is, when a point 12-1 (A in FIG. 1) exists in the voxel 11-2, the point 12-1 is converted to a point 12-2 corresponding to the voxel 11-2 by correcting the position. Note that, although only one voxel 11-1 is marked in B in FIG. 1, all the squares indicated by the dotted lines in the bounding box 10 in B in FIG. 1 are voxels 11-1. Similarly, although only one voxel 11-2 is marked in B in FIG. 1, all the squares indicated by the thick lines in the bounding box 10 in B in FIG. 1 are voxels 11-2. Similarly, although only one point 12-2 is labeled in FIG. 1B, all of the black circles shown in FIG. 1B are points 12-2.

[0025] Similarly, as shown in FIG. 1C, four voxels 11-2 arranged vertically and horizontally are integrated to form a higher-level voxel 11-3 indicated by a thick line. Then, the geometry data is quantized using this voxel 11-3. That is, when a point 12-2 (FIG. 1B) exists in the voxel 11-3, the point 12-2 is converted to a point 12-3 corresponding to the voxel 11-3 by correcting the position. Note that, although only one voxel 11-2 is indicated by a symbol in FIG. 1C, all squares indicated by dotted lines in the bounding box 10 in FIG. 1C are voxels 11-2. Similarly, although only one voxel 11-3 is indicated by a symbol in FIG. 1C, all squares indicated by thick lines in the bounding box 10 in FIG. 1C are voxels 11-3. Similarly, although only one point 12-3 is labeled in FIG. 1C, all of the black circles shown in FIG. 1C are points 12-3.

[0026] Similarly, as shown in FIG. 1D, four voxels 11-3 arranged vertically and horizontally are integrated to form the next higher voxel 11-4 indicated by a thick line. Then, the geometry data is quantized using this voxel 11-4. That is, if a point 12-3 (FIG. 1C) exists in the voxel 11-4, the point 12-3 is converted to a point 12-4 corresponding to the voxel 11-4 by correcting the position. Note that, although only one voxel 11-3 is marked in FIG. 1D, all squares indicated by dotted lines in the bounding box 10 in FIG. 1D are voxels 11-3.

[0027] In this way, the geometry data is organized into a tree structure (octree).

[0028] <Lifting> In contrast, when encoding attribute data, the geometry data, including degradation due to encoding, is considered to be known, and encoding is performed using the positional relationship between points. As a method for encoding such attribute data, a method using a transformation called RAHT (Region Adaptive Hierarchical Transform) or Lifting as described in Non-Patent Document 4 has been considered. By applying these techniques, it is possible to make the reference structure (reference relationship) of attribute data into a hierarchical structure (tree structure) like the octree of geometry data.

[0029] For example, in the case of lifting, the attribute data of each point is encoded as a difference value from a predicted value derived using the attribute data of other points. Then, the points for deriving the difference value (i.e., deriving the predicted value) are selected hierarchically.

[0030] For example, in the hierarchy shown in A of Fig. 2, among the points (P0 to P9) shown by circles, points P7, P8, and P9 shown by white circles are selected as prediction points, which are points from which predicted values ​​are derived, and the other points P0 to P6 are set to be selected as reference points, which are points whose attribute data are referenced when deriving the predicted values. In other words, in this hierarchy, for each of the prediction points P7 to P9, a difference value between the attribute data and the predicted value is derived.

[0031] For the sake of simplicity, the three-dimensional space is illustrated as a two-dimensional plane in Fig. 2. In other words, the points P0 to P6 are actually arranged in a three-dimensional space.

[0032] Each arrow in A of Fig. 2 indicates a reference relationship when deriving a predicted value. For example, the predicted value of prediction point P7 is derived by referring to attribute data of reference points P0 and P1. The predicted value of prediction point P8 is derived by referring to attribute data of reference points P2 and P3. The predicted value of prediction point P9 is derived by referring to attribute data of reference points P4 to P6. Then, for each of prediction points P7 to P9, a difference value between the predicted value calculated as described above and the attribute data is derived.

[0033] In the next higher hierarchical level, as shown in B of Figure 2, the points (P0 to P6) selected as reference points in the hierarchical level A of Figure 2 (the next lower hierarchical level) are classified (sorted) into prediction points and reference points in the same manner as in the case of the hierarchical level A of Figure 2.

[0034] For example, points P1, P3, and P6 indicated by gray circles in B of Fig. 2 are selected as prediction points, and points P0, P2, P4, and P5 indicated by black circles are selected as reference points. That is, in this hierarchy, for each of the prediction points P1, P3, and P6, a difference value between the attribute data and its prediction value is derived.

[0035] Each arrow in B of FIG. 2 indicates a reference relationship when deriving a predicted value. For example, the predicted value of prediction point P1 is derived by referring to the attribute data of reference points P0 and P2. The predicted value of prediction point P3 is derived by referring to the attribute data of reference points P2 and P4. The predicted value of prediction point P6 is derived by referring to the attribute data of reference points P4 and P5. Then, for each of prediction points P1, P3, and P6, a difference value between the predicted value calculated as described above and the attribute data is derived.

[0036] At the next higher level, as shown in C of Figure 2, the points (P0, P2, P4, P5) selected as reference points at level B of Figure 2 (the next lower level) are classified (sorted), predicted values ​​of each prediction point are derived, and difference values ​​between the predicted values ​​and attribute data are derived.

[0037] By repeating this classification recursively for the reference points at the next lower level, the reference structure of the attribute data is made hierarchical.

[0038] <Point classification> The procedure for classifying (sorting) points in such lifting will now be described in more detail. In lifting, points are classified in order from the lower layer to the upper layer, as described above. In each layer, first, the points are sorted in Morton Code order. Next, the first point in the row of points sorted in Morton Code order is selected as the reference point. Next, a search is made for points (neighboring points) located near the reference point, and the searched point (neighboring point) is set as the predicted point (also called index point).

[0039] For example, as shown in Fig. 3, points are searched for within a circle 22 of radius R centered on a reference point 21 to be processed. This radius R is set in advance for each layer. In the example of Fig. 3, points 23-1 to 23-4 are detected and set as predicted points.

[0040] For the sake of simplicity, the three-dimensional space is illustrated as a two-dimensional plane in Fig. 3. In other words, in reality, each point is arranged in a three-dimensional space, and the search for the point is performed within a spherical region of radius R.

[0041] Next, the remaining points are classified in the same way. In other words, among the points that have not been selected as either a reference point or a prediction point at the current time, the first point in the order of the Morton code is selected as the reference point, and points near the reference point are searched for and set as prediction points.

[0042] When the above process is repeated until all points are classified, the process for that layer ends and the processing moves to the next higher layer. The above procedure is then repeated for that layer. In other words, each point selected as a reference point in the next lower layer is sorted in Morton code order, and classified into reference points and prediction points as described above. By repeating the above process, the reference structure of the attribute data is hierarchically organized.

[0043] <Derivation of predicted values> As described above, in the case of lifting, the predicted value of the attribute data of a prediction point is derived using the attribute data of reference points around the prediction point. For example, as shown in Fig. 4, the predicted value of a prediction point Q(i,j) is derived by referring to the attribute data of reference points P1 to P3.

[0044] For the sake of simplicity, the three-dimensional space is illustrated as a two-dimensional plane in Fig. 4. In other words, each point is actually disposed in a three-dimensional space.

[0045] In this case, the attribute data of each reference point is weighted and integrated by a weight value (α(P,Q(i,j))) according to the inverse of the distance (actually the distance in three-dimensional space) between the prediction point and the reference point, as shown in the following formula (1). Here, A(P) indicates the attribute data of point P.

[0046]

number

[0047] In the method described in Non-Patent Document 4, the highest resolution (i.e., the lowest layer) position information is used to derive the distance between the predicted point and its reference point. <Quantization> After the attribute data is layered as described above, it is quantized and encoded. During the quantization, the attribute data (difference value) of each point is weighted according to the layer structure as shown in the example of FIG. 5. This weight value (Quantization Weight) W is derived for each point using the weight value of the lower layer as shown in FIG. 5. This weight value can also be used in lifting (layering of attribute data) to improve compression efficiency.

[0048] <Tree structure inconsistency> In the case of lifting described in Non-Patent Document 4, as described above, the method of hierarchizing the reference structure of attribute data is different from that of tree structuring (e.g., octree) of geometry data. Therefore, it is not guaranteed that the reference structure of attribute data matches the tree structure of geometry data. Therefore, in order to decode attribute data, it is necessary to decode the geometry data up to the lowest layer regardless of its layer. In other words, it is difficult to realize scalable decoding of point cloud data without decoding unnecessary information.

[0049] <Achieving scalable decoding of point cloud data> Therefore, as described in Non-Patent Document 2, a method has been proposed to make the reference structure of attribute data similar to the tree structure of geometry data. More specifically, when constructing the reference structure of attribute data, a prediction point is selected so that a point also exists in a voxel in a layer one level higher than the voxel in which a point in the current layer exists. In this way, it is possible to decode point cloud data at a desired resolution without decoding unnecessary information. In other words, it is possible to realize scalable decoding of point cloud data.

[0050] For example, as shown in A of Fig. 6, in a bounding box 100 which is a predetermined three-dimensional spatial region, points 102-1 to 102-9 are arranged for each voxel 101-1 of a predetermined hierarchy. In Fig. 6, for the sake of simplicity, the three-dimensional space is described as a two-dimensional plane. That is, in reality, the bounding box is a three-dimensional spatial region, and the voxel is a small region of a rectangular parallelepiped (including a cube). The points are arranged in the three-dimensional space.

[0051] When it is not necessary to distinguish between the voxels 101-1 to 101-3, they will be referred to as the voxels 101. When it is not necessary to distinguish between the points 102-1 to 102-9, they will be referred to as the points 102.

[0052] In this hierarchy, points 102-1 to 102-9 are classified into prediction points and reference points so that points also exist in voxel 101-2, which is one hierarchy higher than voxel 101-1 in which points 102-1 to 102-9 exist, as shown in B of Fig. 6. In the example of B of Fig. 6, points 102-3, 102-5, and 102-8, which are indicated by white circles, are set as prediction points, and the other points are set as reference points.

[0053] Similarly, in the next higher hierarchical level, points 102-1, 102-2, 102-4, 102-6, 102-7, and 102-9 are present in voxel 101-2, and voxel 101-3 in the next higher hierarchical level also has points, so these points 102 are classified as prediction points and reference points (C in FIG. 6). In the example in C in FIG. 6, points 102-1, 102-4, and 102-7, indicated by gray circles, are set as prediction points, and the other points are set as reference points.

[0054] By doing this, as shown in D of Fig. 6, a hierarchical structure is created such that voxel 101-3, which has point 102 in a lower layer, has one point 102. This process is performed for each layer. In other words, by performing this process when constructing the reference structure of the attribute data (when classifying prediction points and reference points in each layer), the reference structure of the attribute data can be made similar to the tree structure (octree) of the geometry data.

[0055] Decoding is performed in the reverse order of FIG. 6, for example, as shown in FIG. 7. For example, as shown in A of FIG. 7, in a given bounding box 100, a point 102-2, a point 102-6, and a point 102-9 are arranged for each voxel 101-3 of a given hierarchy (similar to the state shown in D of FIG. 6). Note that, in FIG. 7 as well, for the sake of simplicity, the three-dimensional space is described as a two-dimensional plane. That is, in reality, the bounding box is a three-dimensional spatial region, and the voxel is a small region of a rectangular parallelepiped (including a cube). The points are arranged in the three-dimensional space.

[0056] At the next lower level, as shown in FIG. 6B, attribute data for points 102-2, 102-6, and 102-9 for each voxel 101-3 are used to derive predicted values ​​for points 102-1, 102-4, and 102-7 and add them to the difference value to restore the attribute data for point 102 for each voxel 101-2 (the same state as FIG. 6C).

[0057] Furthermore, at the next lower level, as shown in C of FIG. 7, predicted values ​​for points 102-3, 102-5, and 102-8 are derived using attribute data for points 102-1, 102-2, 102-4, 102-6, 102-7, and 102-9 for each voxel 101-2, and these are added to the difference value to restore the attribute data (similar to the state in B of FIG. 6).

[0058] By doing this, the attribute data of the point 102 for each voxel 101-1 is restored as shown in D of Fig. 7 (similar to A of Fig. 6). In other words, like the case of an octree, the attribute data of each layer can be restored using the attribute data of the upper layer.

[0059] In this way, the reference structure (hierarchical structure) of the attribute data can be associated with the tree structure (hierarchical structure) of the geometry data. Therefore, since the geometry data corresponding to each attribute data can be obtained even at the intermediate resolution, the geometry data and the attribute data can be correctly decoded at the intermediate resolution. In other words, scalable decoding of the point cloud data can be realized.

[0060] <Geometry Scaling> Meanwhile, as described in Non-Patent Document 3, a method called Geometry Scaling has been proposed in which geometry data is scaled and points are thinned out when encoding a point cloud. In geometry scaling, the geometry data of nodes is quantized during encoding. This process makes it possible to thin out points according to the characteristics of the area to be encoded.

[0061] For example, as shown in A of Fig. 8, six points (point1 to point6) with X coordinates of 100, 101, 102, 103, 104, and 105 are processed. Note that, for simplicity, only the X coordinate will be described here. That is, since points are actually placed in a three-dimensional space, the same processing as for the X coordinate described below is also performed for the Y coordinate and Z coordinate.

[0062] When geometry scaling is applied to these six points, the X coordinate of each point is scaled according to the value of the quantization parameter baseQP, as shown in the table in FIG. 8B. When baseQP=0, no scaling is performed, so the X coordinate of each point remains as shown in FIG. 8A. For example, when baseQP=4, the X coordinate of point2 is scaled from 101 to 102, the X coordinate of point4 is scaled from 103 to 104, and the X coordinate of point6 is scaled from 105 to 106.

[0063] When the X coordinates of multiple points overlap due to this scaling, they can be merged. For example, when mergeDuplicatePoint=1, such duplicate points are merged into one point. mergeDuplicatePoint is flag information indicating whether or not such duplicate points are merged. That is, such merging thins out points (reduces the number of points). For example, in each row of the table in B of FIG. 8, such merging can thin out the points shown in gray. For example, when baseQP=4, the number of points for the four points (point2 to point5) shown in thick lines is reduced by half.

[0064] In order to realize scalable decoding of point cloud data, when attribute data is hierarchized as described in Non-Patent Document 2, a tree structure of the geometry data is estimated based on the decoding result of the encoded data of the geometry data. In other words, a reference structure of the attribute data is formed so that it is similar to the estimated tree structure of the geometry data.

[0065] <Tree structure inconsistency due to geometry scaling> However, when geometry scaling is applied as described above, there is a possibility that points are thinned out in the decoded result of the encoded data of the geometry data. If points are thinned out, it may be impossible to estimate the tree structure of the actual geometry data. In other words, there is a possibility that a mismatch may occur between the estimated tree structure and the actual tree structure (the tree structure corresponding to the points before thinning out). In such a case, there is a possibility that scalable decoding of the point cloud data may not be realized.

[0066] An example will be described for the case where baseQP = 4 in B of Fig. 8. It is assumed that the tree structure corresponding to each point (point1 to point6) of the table shown in A of Fig. 8 is the tree structure shown in Fig. 9. Nodes 121-1 to 121-6 of this tree structure correspond to each point (point1 to point6) of the table shown in A of Fig. 8. In other words, it is assumed that the X coordinates of nodes 121-1 to 121-6 in the lowest layer (third layer) of this tree structure are 100, 101, 102, 103, 104, and 105, respectively.

[0067] The following two rules are applied to the tree structure in Figure 9. The first rule is that in the layer one level above the third layer (the second layer), the nodes with X coordinates of 100 and 101, the nodes with X coordinates of 102 and 103, and the nodes with X coordinates of 104 and 105 in the third layer are grouped together. The second rule is that in the top layer (the first layer), all nodes in the second layer are grouped together.

[0068] That is, nodes 121-1 and 121-2 in the third layer belong to node 122-1 in the second layer. Nodes 121-3 and 121-4 in the third layer belong to node 122-2 in the second layer. Nodes 121-5 and 121-6 in the third layer belong to node 122-3 in the second layer. Also, nodes 122-1 to 122-3 in the second layer belong to node 123 in the first layer.

[0069] When the geometry scaling is performed, the X coordinate of node 121-2 is scaled from 101 to 102, the X coordinate of node 121-4 is scaled from 103 to 104, and the X coordinate of node 121-6 is scaled from 105 to 106. As a result, the X coordinates of node 121-2 and node 121-3 overlap, so they are merged and node 121-3 is thinned out. Similarly, node 121-4 and node 121-5 are merged and node 121-5 is thinned out. As a result, in the third layer, node 121-1, node 121-2, node 121-4, and node 121-6 are encoded.

[0070] In encoding attribute data, the highest resolution geometry data is obtained as a result of decoding the encoded data of the geometry data. That is, in the example of Fig. 9, nodes 121-1, 121-2, 121-4, and 121-6 are obtained. Then, the tree structure of the geometry data is estimated from these four points (four nodes).

[0071] In this case, in order to form a similar tree structure, the same rules as in the example of Fig. 9 are applied. That is, in the layer one level above the third layer (second layer), the nodes with X coordinates of 100 and 101, the nodes with X coordinates of 102 and 103, and the nodes with X coordinates of 104 and 105 in the third layer are grouped together. Also, in the top layer (first layer), all the nodes in the second layer are grouped together.

[0072] As a result, a tree structure as shown in Fig. 10 is estimated. That is, node 121-1 in the third layer belongs to node 124-1 in the second layer. Node 121-2 in the third layer belongs to node 124-2 in the second layer. Node 121-4 in the third layer belongs to node 124-3 in the second layer. Node 121-6 in the third layer belongs to node 124-4 in the second layer. Also, nodes 124-1 to 124-4 in the second layer belong to node 125 in the first layer.

[0073] As is clear from a comparison of FIG. 9 and FIG. 10, these tree structures do not match. For example, assume that the value of the parameter skipOctreeLayer indicating the layer to be decoded is "1" (skipOctreeLayer = 1). That is, when the second layer is decoded, three nodes are obtained for the geometry data as shown in FIG. 9. In contrast, four nodes are obtained for the attribute data as shown in FIG. 10. In this way, due to the mismatch of the tree structures, it is difficult to obtain the decoded result of the desired layer without decoding unnecessary information. In other words, it is difficult to realize scalable decoding of point cloud data.

[0074] In other words, in order to realize scalable decoding of point cloud data, it is necessary to estimate the tree structure taking into account such geometry scaling. In other words, complicated processing such as changing the tree structure estimation method depending on whether geometry scaling is performed or not is required. In addition, there is a risk of cost increase due to the preparation of multiple estimation methods.

[0075] In addition, in geometry scaling, the method of thinning out overlapping points (which points to thin out among multiple overlapping points) is not specified and depends on the design. In other words, a method of estimating a tree structure corresponding to geometry scaling must be newly designed according to the design of the geometry scaling, which may increase costs.

[0076] <Encoding method restrictions> Therefore, as shown in the top row of the table in FIG. 11, applicable encoding methods are restricted, and the combined use of scalable decoding encoding and processing for updating the tree structure of geometry data is prohibited.

[0077] In other words, in encoding point clouds, which represent three-dimensional objects as a collection of points, control is provided to prohibit the combined use of scalable encoding, an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, an encoding method that involves modifying the tree structure of geometry data.

[0078] For example, in an information processing device, an encoding control unit is provided that performs control to prohibit the combined use of scalable encoding, which is an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, which is an encoding method that involves changing the tree structure of geometry data, in encoding a point cloud that represents a three-dimensional object as a collection of points.

[0079] In other words, when scalable coding is applied, the application of scaling coding is prohibited, and when scaling coding is applied, the application of scalable coding is prohibited. In this way, when scalable coding is performed, it is possible to suppress the occurrence of a mismatch between the reference structure of attribute data and the tree structure of geometry data. Therefore, it is possible to more easily realize scalable decoding of point cloud data.

[0080] The scalable coding may be any coding method that generates coded data that can be decoded in a scalable manner. For example, the scalable coding may be lifting scalability, which encodes attribute data by lifting the data using a reference structure similar to the tree structure of geometry data, as described in Non-Patent Document 2. In other words, the coding may be controlled to prohibit the combined use of lifting scalability and scaling coding.

[0081] Furthermore, the scaling coding may be any coding method that involves changing the tree structure of the geometry data. For example, it may be geometry scaling, which scales and codes the geometry data, as described in Non-Patent Document 3. In other words, it may be controlled to prohibit the combined use of scalable coding and geometry scaling.

[0082] Of course, as shown in the second row from the top of the table in FIG. 11, the combined use of Lifting Scalability and Geometry Scaling may be prohibited (Method 1).

[0083] In this case, the coding control unit may control to prohibit application of scaling coding when applying scalable coding. For example, as shown in the third row from the top of the table in FIG. 11, when applying lifting scalability, application of geometry scaling may be prohibited (method 1-1).

[0084] In addition, the coding control unit may perform control so as to prohibit application of scalable coding when applying scaling coding. For example, as shown in the fourth row from the top of the table in FIG. 11, when applying geometry scaling, application of lifting scalability may be prohibited (method 1-2).

[0085] In order to perform such control, for example, flag information (permission flag or prohibition flag) indicating whether or not to permit (or prohibit) the application of scalable coding and scaling coding may be set, as shown in the fifth row from the top of the table in FIG. 11 (Method 2).

[0086] For example, the encoding control unit may control the signaling of a scalable encoding enable flag, which is flag information regarding the application of scalable encoding, and a scaling encoding enable flag, which is flag information regarding the application of scaling encoding.

[0087] For example, as described in the sixth row from the top of the table shown in Fig. 11, such a restriction may be specified in the semantics (method 2-1). For example, in the semantics, when a scalable coding enable flag with a value indicating application of scalable coding is signaled, a scaling coding enable flag with a value indicating non-application of scaling coding may be specified to be signaled, and the coding control unit may perform signaling according to the semantics. Also, in the semantics, when a scaling coding enable flag with a value indicating application of scaling coding is signaled, a scalable coding enable flag with a value indicating non-application of scalable coding may be specified to be signaled, and the coding control unit may perform signaling according to the semantics.

[0088] FIG. 12 shows an example of semantics in this case. For example, the semantics 161 shown in FIG. 12 specifies that if the value of geom_scaling_enabled_flag is greater than 0, the value of lifting_scalability_enabled_flag must be set to 0. Here, geom_scaling_enabled_flag is flag information indicating whether or not geometry scaling is applied. If geom_scaling_enabled_flag = 1, geometry scaling is applied. Also, if geom_scaling_enabled_flag = 0, geometry scaling is not applied. lifting_scalability_enabled_flag is flag information indicating whether or not lifting scalability is applied. If lifting_scalability_enabled_flag = 1, lifting scalability is applied. Also, if lifting_scalability_enabled_flag = 0, lifting scalability is not applied.

[0089] That is, in this semantics 161, when geometry scaling is applied, the application of lifting scalability is prohibited. Conversely, in the semantics, when lifting scalability is applied, the application of geometry scaling may be prohibited. That is, in the semantics, it may be specified that when the value of lifting_scalability_enabled_flag is greater than 0, the value of geom_scaling_enabled_flag must be 0.

[0090] Also, for example, as shown in the seventh row from the top of the table shown in Fig. 11, signaling based on such restrictions may be performed in the profile (method 2-2). For example, when scalable coding is applied, the coding control unit may control to signal, in the profile, a scalable coding enable flag having a value indicating application of scalable coding and a scaling coding enable flag having a value indicating non-application of scaling coding. Also, when scaling coding is applied, the coding control unit may control to signal, in the profile, a scaling coding enable flag having a value indicating application of scaling coding and a scalable coding enable flag having a value indicating non-application of scalable coding.

[0091] An example of a profile when lifting scalability is applied is shown in Fig. 13. For example, in the profile 162 shown in Fig. 13, lifting_scalability_enabled_flag = 1 and geom_scaling_enabled_flag = 0 are signaled. That is, in this profile 162, it is indicated that lifting scalability is applied and geometry scaling is not applied.

[0092] In addition, in a profile when geometry scaling is applied, geom_scaling_enabled_flag = 1 and lifting_scalability_enabled_flag = 0 may be signaled.

[0093] Furthermore, such a restriction may be specified in the syntax, for example, as described in the bottom row of the table shown in Fig. 11 (Method 2-3). For example, when a scalable coding enable flag having a value indicating the application of scalable coding is signaled, the coding control unit may control to omit the signaling of the scaling coding enable flag according to the above-mentioned syntax. Also, when a scaling coding enable flag having a value indicating the application of scaling coding is signaled, the coding control unit may control to omit the signaling of the scalable coding enable flag according to the above-mentioned syntax.

[0094] An example of the syntax in this case is shown in Fig. 14. For example, in syntax 163 shown in Fig. 14, lifting_scalability_enabled_flag is signaled only when geom_scaling_enabled_flag is not signaled (i.e., the value is set to "0" and geometry scaling is not applied). That is, in this case, lifting scalability can be applied. In other words, when geometry scaling is applicable (i.e., when geom_scaling_enabled_flag is signaled), lifting_scalability_enabled_flag is not signaled (i.e., the value is set to "0" and lifting scalability is not applied).

[0095] Conversely, geom_scaling_enabled_flag may be signaled only when lifting_scalability_enabled_flag is not signaled (i.e., the value is set to "0" and lifting scalability is not applied). In other words, in this case, geometry scaling can be applied. In other words, when lifting scalability is applicable (i.e., when lifting_scalability_enabled_flag is signaled), geom_scaling_enabled_flag may not be signaled (i.e., the value is set to "0" and geometry scaling is not applied).

[0096] <2. First embodiment> <Encoding device> Next, a device to which the present technology described above in <1. Encoding control> is applied will be described. Fig. 15 is a block diagram showing an example of a configuration of an encoding device, which is one aspect of an information processing device to which the present technology is applied. The encoding device 200 shown in Fig. 15 is a device that encodes a point cloud (3D data). The encoding device 200 encodes the point cloud by applying the present technology described above in <1. Encoding control>.

[0097] Note that Fig. 15 shows the main processing units, data flows, etc., and is not limited to all that is shown in Fig. 15. In other words, in encoding device 200, there may be processing units that are not shown as blocks in Fig. 15, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 15.

[0098] As shown in FIG. 15, the encoding device 200 has an encoding control unit 201, a geometry data encoding unit 211, a geometry data decoding unit 212, a point cloud generating unit 213, an attribute data encoding unit 214, and a bit stream generating unit 215.

[0099] The encoding control unit 201 performs processing related to control of encoding of point cloud data. For example, the encoding control unit 201 controls the geometry data encoding unit 211. Also, the encoding control unit 201 controls the attribute data encoding unit 214. For example, as described above in <1. Encoding control>, the encoding control unit 201 controls these processing units so as to prohibit the combined use of scalable encoding, which is an encoding method that generates encoded data that can be scalably decoded, and scaling encoding, which is an encoding method that involves changing the tree structure of geometry data. Also, the encoding control unit 201 controls the bit stream generation unit 215, and controls signaling between a scalable encoding enable flag (e.g., Lifting_scalability_enabled_flag) that is flag information regarding the application of scalable encoding, and a scaling encoding enable flag (e.g., geom_scaling_enabled_flag) that is flag information regarding the application of scaling encoding.

[0100] The geometry data encoding unit 211 encodes geometry data (position information) of the point cloud (3D data) input to the encoding device 200, and generates the encoded data. Any encoding method may be used. For example, filtering, quantization, or other processing for noise suppression (denoising) may be performed. However, the geometry data encoding unit 211 performs this encoding under the control of the encoding control unit 201. That is, in this encoding, the geometry data encoding unit 211 applies geometry scaling under the control of the encoding control unit 201. The geometry data encoding unit 211 supplies the generated encoded data of the geometry data to the geometry data decoding unit 212 and the bit stream generation unit 215.

[0101] The geometry data decoding unit 212 obtains the coded data of the geometry data supplied from the geometry data encoding unit 211, and decodes the coded data. Any decoding method may be used as long as it is compatible with the coding performed by the geometry data encoding unit 211. For example, processing such as filtering and inverse quantization for denoising may be performed. The geometry data decoding unit 212 supplies the generated geometry data (decoded result) to the point cloud generating unit 213.

[0102] The point cloud generation unit 213 acquires attribute data (attribute information) of the point cloud input to the encoding device 200 and geometry data (decoded result) supplied from the geometry data decoding unit 212. The point cloud generation unit 213 performs processing (recolor processing) to make the attribute data correspond to the geometry data (decoded result). The point cloud generation unit 213 supplies the attribute data that has been made to correspond to the geometry data (decoded result) to the attribute data encoding unit 214.

[0103] The attribute data encoding unit 214 acquires the geometry data (decoded result) and attribute data supplied from the point cloud generation unit 213. The attribute data encoding unit 214 uses the geometry data (decoded result) to encode the attribute data and generate encoded data of the attribute data. However, the attribute data encoding unit 214 performs this encoding under the control of the encoding control unit 201. That is, in this encoding, the attribute data encoding unit 214 applies lifting scalability under the control of the encoding control unit 201. The attribute data encoding unit 214 supplies the generated encoded data of the attribute data to the bit stream generation unit 215.

[0104] The bitstream generating unit 215 obtains the coded data of the geometry data supplied from the geometry data encoding unit 211. The bitstream generating unit 215 also obtains the coded data of the attribute data supplied from the attribute data encoding unit 214. The bitstream generating unit 215 generates a bitstream including these coded data. The bitstream generating unit 215 also performs signaling of control information such as a scalable coding enable flag (e.g., Lifting_scalability_enabled_flag) which is flag information regarding the application of scalable coding and a scaling coding enable flag (e.g., geom_scaling_enabled_flag) which is flag information regarding the application of scaling coding (includes the control information in the bitstream) according to the control of the coding control unit 201. The bitstream generating unit 215 outputs the generated bitstream to the outside of the coding device 200 (e.g., the decoding side).

[0105] With this configuration, the encoding device 200 can prohibit the combined use of scalable encoding, which is an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, which is an encoding method that involves changing the tree structure of geometry data, making it easier to achieve scalable decoding of point cloud data.

[0106] These processing units (the encoding control unit 201, the geometry data encoding unit 211 to the bit stream generation unit 215) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and by executing a program.

[0107] <Geometry data encoding part> Fig. 16 is a block diagram showing a main configuration example of the geometry data encoding unit 211. Note that Fig. 16 shows main processing units, data flows, etc., and is not limited to all that are shown in Fig. 16. In other words, in the geometry data encoding unit 211, there may be processing units that are not shown as blocks in Fig. 16, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 16.

[0108] As shown in FIG. 16, the geometry data encoding unit 211 includes a voxel generation unit 231 , a tree structure generation unit 232 , a selection unit 233 , a geometry scaling unit 234 , and an encoding unit 235 .

[0109] The voxel generation unit 231 performs processing related to the generation of voxel data. For example, the voxel generation unit 231 sets a bounding box for the input point cloud data, and sets voxels to divide the bounding box. Then, the voxel generation unit 231 quantizes the geometry data of each point in units of voxels to generate voxel data. The voxel generation unit 231 supplies the generated voxel data to the tree structure generation unit 232.

[0110] The tree structure generating unit 232 performs processing related to the generation of a tree structure. For example, the tree structure generating unit 232 acquires voxel data supplied from the voxel generating unit 231. In addition, the tree structure generating unit 232 converts the voxel data into a tree structure. For example, the tree structure generating unit 232 generates an octree using the voxel data. The tree structure generating unit 232 supplies data of the generated octree to the selecting unit 233.

[0111] The selection unit 233 performs processing related to control of application / non-application of geometry scaling. For example, the selection unit 233 acquires octree data supplied from the tree structure generation unit 232. Furthermore, the selection unit 233 selects a supply destination of the octree data in accordance with the control of the encoding control unit 201. That is, the selection unit 233 selects whether to supply the octree data to the geometry scaling unit 234 or to the encoding unit 235 in accordance with the control of the encoding control unit 201, and supplies the octree data to the selected supply destination.

[0112] For example, when the encoding control unit 201 instructs application of geometry scaling, the selection unit 233 supplies the octree data to the geometry scaling unit 234. On the other hand, when the encoding control unit 201 instructs not to apply geometry scaling, the selection unit 233 supplies the octree data to the encoding unit 235.

[0113] The geometry scaling unit 234 performs processing related to geometry scaling. For example, the geometry scaling unit 234 acquires octree data supplied from the selection unit 233. The geometry scaling unit 234 also performs geometry scaling on the octree data, scaling the geometry data, and merging points. The geometry scaling unit 234 supplies the octree data subjected to the geometry scaling to the encoding unit 235.

[0114] The encoding unit 235 performs processing related to encoding of octree data (octree-arranged voxel data (i.e., geometry data)). For example, the encoding unit 235 acquires octree data supplied from the selection unit 233 or the geometry scaling unit 234. For example, when the encoding control unit 201 instructs the encoding unit 235 to apply geometry scaling, the encoding unit 235 acquires octree data to which geometry scaling has been applied, supplied from the geometry scaling unit. Also, when the encoding control unit 201 instructs the encoding unit 235 not to apply geometry scaling, the encoding unit 235 acquires octree data to which geometry scaling has not been applied, supplied from the selection unit 233.

[0115] The encoding unit 235 encodes the acquired octree data to generate encoded data of the geometry data. Any encoding method may be used. The encoding unit 235 supplies the generated encoded data of the geometry data to the geometry data decoding unit 212 and the bitstream generating unit 215 (both shown in FIG. 15).

[0116] <Attribute data encoding part> Fig. 17 is a block diagram showing a main configuration example of the attribute data encoding unit 214. Note that Fig. 17 shows main processing units, data flows, etc., and is not limited to all that are shown in Fig. 17. In other words, in the attribute data encoding unit 214, there may be processing units that are not shown as blocks in Fig. 17, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 17.

[0117] As shown in FIG. 17, the attribute data encoding unit 214 has a selection unit 251, a scalable layer processing unit 252, a layer processing unit 253, a quantization unit 254, and an encoding unit 255.

[0118] The selection unit 251 performs processing related to control of application / non-application of lifting scalability. For example, the selection unit 251 selects a supply destination of attribute data, geometry data (decoded results), and the like generated by the point cloud generation unit 213 (FIG. 15) in accordance with the control of the encoding control unit 201. That is, the selection unit 251 selects whether to supply the data to the scalable hierarchical processing unit 252 or to the hierarchical processing unit 253 in accordance with the control of the encoding control unit 201, and supplies the data to the selected supply destination.

[0119] For example, when the encoding control unit 201 instructs application of lifting scalability, the selection unit 251 supplies attribute data, geometry data (decoded results), etc. to the scalable hierarchical processing unit 252. On the other hand, when the encoding control unit 201 instructs not to apply lifting scalability, the selection unit 251 supplies attribute data, geometry data (decoded results), etc. to the hierarchical processing unit 253.

[0120] The scalable hierarchical processing unit 252 performs processing related to lifting of attribute data (formation of a reference structure). For example, the scalable hierarchical processing unit 252 acquires attribute data and geometry data (decoding result) supplied from the selection unit 251. The scalable hierarchical processing unit 252 uses the geometry data to hierarchicalize the attribute data (i.e., form a reference structure). At this time, the scalable hierarchical processing unit 252 performs hierarchicalization by applying the method described in Non-Patent Document 2. That is, the scalable hierarchical processing unit 252 estimates a tree structure (octree) based on the geometry data, forms a reference structure of the attribute data so as to correspond to the estimated tree structure, derives a predicted value according to the reference structure, and derives a difference value between the predicted value and the attribute data. The scalable hierarchical processing unit 252 supplies the attribute data (difference value) thus generated to the quantization unit 254.

[0121] The hierarchical processing unit 253 performs processing related to lifting of attribute data (formation of a reference structure). For example, the hierarchical processing unit 253 acquires attribute data and geometry data (decoding result) supplied from the selection unit 251. The hierarchical processing unit 253 uses the geometry data to hierarchically organize the attribute data (i.e., form a reference structure). At this time, the hierarchical processing unit 253 performs hierarchical organization by applying the method described in Non-Patent Document 4. That is, the hierarchical processing unit 253 forms a reference structure of the attribute data independently of the tree structure (octree) of the geometry data, derives a predicted value according to the reference structure, and derives a difference value between the predicted value and the attribute data. That is, the hierarchical processing unit 253 does not estimate the tree structure of the geometry data. The hierarchical processing unit 253 supplies the attribute data (difference value) generated in this manner to the quantization unit 254.

[0122] The quantization unit 254 acquires attribute data (difference value) supplied from the scalable hierarchical processing unit 252 or the hierarchical processing unit 253. The quantization unit 254 quantizes the attribute data (difference value). The quantization unit 254 supplies the quantized attribute data (difference value) to the encoding unit 255.

[0123] The encoding unit 255 acquires the quantized attribute data (difference value) supplied from the quantization unit 254. The encoding unit 255 encodes the quantized attribute data (difference value) to generate encoded data of the attribute data. Any encoding method may be used. The encoding unit 255 supplies the generated encoded data of the attribute data to the bit stream generation unit 215 (FIG. 15).

[0124] With the above-mentioned configuration, the encoding device 200 can prohibit the combined use of scalable encoding, which is an encoding method that generates scalably decodable encoded data, and scaling encoding, which is an encoding method that involves changing the tree structure of geometry data, in encoding a point cloud that represents a three-dimensional object as a set of points. Therefore, for example, in encoding attribute data, scalable decoding of point cloud data can be realized without the need for complicated processing such as preparing a plurality of methods for estimating the tree structure of geometry data and selecting one from among them. In addition, since it is not necessary to newly design the method for estimating the tree structure, an increase in cost can be suppressed. In other words, scalable decoding of point cloud data can be realized more easily.

[0125] <Flow of encoding control process> Next, a description will be given of the processing executed by the encoding device 200. The encoding control unit 201 of the encoding device 200 controls the encoding of the point cloud data by executing an encoding control processing. An example of the flow of the encoding control processing will be described with reference to the flowchart in FIG.

[0126] When the encoding control process starts, the encoding control unit 201 determines whether or not to apply lifting scalability in step S101. If it is determined that lifting scalability is to be applied, the process proceeds to step S102.

[0127] In step S102, the encoding control unit 201 prohibits geometry scaling in encoding the geometry data. In addition, in step S103, the encoding control unit 201 applies lifting scalability in encoding the attribute data. In addition, as described above in <1. Encoding control>, the encoding control unit 201 signals control information such as Lifting_scalability_enabled_flag and geom_scaling_enabled_flag to correspond to these controls.

[0128] When the process of step S103 ends, the encoding control process ends.

[0129] Also, if it is determined in step S101 that lifting scalability is not to be applied, the process proceeds to step S104.

[0130] In step S104, the encoding control unit 201 determines whether or not to apply geometry scaling. If it is determined that geometry scaling is to be applied, the process proceeds to step S105.

[0131] In step S105, the encoding control unit 201 applies geometry scaling in encoding the geometry data. In addition, in step S106, the encoding control unit 201 prohibits lifting scalability in encoding the attribute data. In addition, as described above in <1. Encoding control>, the encoding control unit 201 signals control information such as Lifting_scalability_enabled_flag and geom_scaling_enabled_flag to correspond to these controls.

[0132] When the process of step S106 ends, the encoding control process ends.

[0133] Also, if it is determined in step S104 that geometry scaling is not to be applied, the process proceeds to step S107.

[0134] In step S107, the encoding control unit 201 prohibits geometry scaling in encoding the geometry data. In addition, in step S108, the encoding control unit 201 prohibits lifting scalability in encoding the attribute data. In addition, as described above in <1. Encoding control>, the encoding control unit 201 signals control information such as Lifting_scalability_enabled_flag and geom_scaling_enabled_flag to correspond to these controls.

[0135] When the process of step S108 ends, the encoding control process ends.

[0136] By executing the encoding control process as described above, the encoding control unit 201 can more easily realize scalable decoding of point cloud data.

[0137] <Encoding process flow> The encoding device 200 encodes the point cloud data by executing an encoding process. An example of the flow of this encoding process will be described with reference to the flowchart in FIG.

[0138] When the encoding process is started, in step S201, the geometry data encoding unit 211 of the encoding device 200 executes a geometry data encoding process to encode the geometry data of the input point cloud and generate encoded data of the geometry data.

[0139] In step S202, the geometry data decoding unit 212 decodes the encoded data of the geometry data generated in step S201, and generates geometry data (decoded result).

[0140] In step S203, the point cloud generating unit 213 performs recolor processing using the attribute data of the input point cloud and the geometry data (decoded result) generated in step S202, and makes the attribute data correspond to the geometry data.

[0141] In step S204, the attribute data encoding unit 214 executes an attribute data encoding process to encode the attribute data that has been recolored in step S203, and generates encoded data of the attribute data.

[0142] In step S205, the bit stream generating unit 215 generates and outputs a bit stream including the coded data of the geometry data generated in step S201 and the coded data of the attribute data generated in step S204.

[0143] When the process of step S205 ends, the encoding process ends.

[0144] <Geometry data encoding process flow> Next, an example of the flow of the geometry data encoding process executed in step S201 in FIG. 19 will be described with reference to the flowchart in FIG.

[0145] When the geometry data encoding process is started, the voxel generation unit 231 of the geometry data encoding unit 211 generates voxel data in step S221.

[0146] In step S222, the tree structure generating unit 232 generates a tree structure (octree) of the geometry data using the voxel data generated in step S221.

[0147] In step S223, the selection unit 233 determines whether or not to perform geometry scaling under the control of the encoding control unit 201. If it is determined that geometry scaling is to be performed, the process proceeds to step S22.

[0148] In step S224, the geometry scaling unit 234 performs geometry scaling on the tree-structured geometry data (i.e., octree data) generated in step S222. When the process of step S224 ends, the process proceeds to step S225. Also, if it is determined in step S223 that geometry scaling is not to be performed, the process of step S224 is skipped and the process proceeds to step S225.

[0149] In step S225, the encoding unit 235 encodes the tree-structured geometry data (i.e., octree data) generated in step S222, or the geometry data (i.e., octree data) subjected to geometry scaling in step S224, to generate encoded data of the geometry data.

[0150] When the process of step S225 ends, the geometry data encoding process ends, and the process returns to FIG.

[0151] <Attribute data encoding process flow> Next, an example of the flow of the attribute data encoding process executed in step S204 in FIG. 19 will be described with reference to the flowchart in FIG.

[0152] When the attribute data encoding process is started, in step S241, the selection unit 251 of the attribute data encoding unit 214 determines whether or not to apply lifting scalability under the control of the encoding control unit 201. If it is determined that lifting scalability is applied, the process proceeds to step S242.

[0153] In step S242, the scalable hierarchical processing unit 252 performs lifting using the method described in Non-Patent Document 2. That is, the scalable hierarchical processing unit 252 estimates a tree structure of the geometry data, and hierarchizes the attribute data (forms a reference structure) according to the estimated tree structure. Then, the scalable hierarchical processing unit 252 derives a predicted value according to the reference structure, and derives a difference value between the attribute data and the predicted value.

[0154] When the process of step S242 ends, the process proceeds to step S244. If it is determined in step S241 that lifting scalability is not to be applied, the process proceeds to step S243.

[0155] In step S243, the hierarchical processing unit 253 performs lifting using the method described in Non-Patent Document 4. That is, the hierarchical processing unit 253 performs hierarchical processing of the attribute data (formation of a reference structure) independently of the tree structure of the geometry data. The hierarchical processing unit 253 then derives a predicted value according to the reference structure, and derives a difference value between the attribute data and the predicted value. When the process of step S243 ends, the process proceeds to step S244.

[0156] In step S244, the quantization unit 254 performs a quantization process to quantize each of the difference values ​​derived in step S242 or step S243.

[0157] In step S245, the encoding unit 255 encodes the difference value quantized in step S244 to generate encoded data of the attribute data. When the process of step S245 ends, the attribute data encoding process ends, and the process returns to FIG.

[0158] By performing each process as described above, the encoding device 200 can more easily realize scalable decoding of point cloud data.

[0159] <3. Second embodiment> <Decryption device> Fig. 22 is a block diagram showing an example of a configuration of a decoding device, which is one aspect of an information processing device to which the present technology is applied. The decoding device 300 shown in Fig. 22 is a device that decodes encoded data of a point cloud (3D data). The decoding device 300 decodes the encoded data of the point cloud generated in the encoding device 200, for example.

[0160] Note that Fig. 22 shows the main processing units, data flows, and the like, and is not limited to all that is shown in Fig. 22. In other words, in the decoding device 300, there may be processing units that are not shown as blocks in Fig. 22, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 22.

[0161] As shown in FIG. 22, the decoding device 300 includes an encoded data extraction unit 311, a geometry data decoding unit 312, an attribute data decoding unit 313, and a point cloud generation unit 314.

[0162] The coded data extraction unit 311 acquires and holds a bit stream input to the decoding device 300. The coded data extraction unit 311 extracts coded data of geometry data and attribute data from the top to a desired layer from the bit stream it holds. If the coded data supports scalable decoding, the coded data extraction unit 311 can extract coded data up to intermediate layers. If the coded data does not support scalable decoding, the coded data extraction unit 311 extracts coded data of all layers.

[0163] The encoded data extraction unit 311 supplies the extracted encoded data of the geometry data to a geometry data decoding unit 312. The encoded data extraction unit 311 supplies the extracted encoded data of the attribute data to an attribute data decoding unit 313.

[0164] The geometry data decoding unit 312 acquires the coded data of the position information supplied from the coded data extraction unit 311. The geometry data decoding unit 312 decodes the coded data of the geometry data by performing a reverse process of the geometry data coding process performed by the geometry data coding unit 211 of the coding device 200, and generates geometry data (decoded result). The geometry data decoding unit 312 supplies the generated geometry data (decoded result) to the attribute data decoding unit 313 and the point cloud generation unit 314.

[0165] The attribute data decoding unit 313 obtains the coded data of the attribute data supplied from the coded data extraction unit 311. The attribute data decoding unit 313 obtains the geometry data (decoded result) supplied from the geometry data decoding unit 312. The attribute data decoding unit 313 performs the inverse process of the attribute data coding process performed by the attribute data coding unit 214 of the coding device 200, thereby decoding the coded data of the attribute data using the geometry data (decoded result) and generating attribute data (decoded result). The attribute data decoding unit 313 supplies the generated attribute data (decoded result) to the point cloud generation unit 314.

[0166] The point cloud generation unit 314 obtains geometry data (decoding result) supplied from the geometry data decoding unit 312. The point cloud generation unit 314 obtains attribute data (decoding result) supplied from the attribute data decoding unit 313. The point cloud generation unit 314 generates a point cloud (decoding result) using the geometry data (decoding result) and the attribute data (decoding result). The point cloud generation unit 314 outputs the generated point cloud (decoding result) data to the outside of the decoding device 300.

[0167] With the above-described configuration, the decoding device 300 can correctly decode the coded data of the point cloud data generated by the coding device 200. That is, scalable decoding of the point cloud data can be more easily realized.

[0168] In addition, these processing units (the encoded data extraction unit 311 to the point cloud generation unit 314) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.

[0169] <Decryption process flow> Next, a description will be given of the processing executed by the decoding device 300. The decoding device 300 decodes the encoded data of the point cloud by executing a decoding process. An example of the flow of this decoding process will be described with reference to the flowchart in FIG.

[0170] When the decoding process is started, in step S301, the coded data extraction unit 311 of the decoding device 300 acquires and holds a bitstream, and extracts coded data of geometry data and attribute data up to the LoD depth to be decoded.

[0171] In step S302, the geometry data decoding unit 312 decodes the encoded data of the geometry data extracted in step S301, and generates geometry data (decoded result).

[0172] In step S303, the attribute data decoding unit 313 decodes the coded data of the attribute data extracted in step S301, and generates attribute data (decoded result).

[0173] In step S304, the point cloud generating unit 314 generates and outputs a point cloud (decoding result) using the geometry data (decoding result) generated in step S302 and the attribute data (decoding result) generated in step S303.

[0174] When the process of step S304 ends, the decoding process ends.

[0175] By performing the processes at each step in this manner, the decoding device 300 can correctly decode the coded data of the point cloud data generated by the coding device 200. In other words, scalable decoding of the point cloud data can be more easily realized.

[0176] <4. Notes> <Hierarchization / reverse hierarchy method> In the above, lifting has been described as an example of a method for creating a hierarchy or de-hierarchy for attribute data. However, the method for creating a hierarchy or de-hierarchy for attribute data may be a method other than lifting, such as RAHT.

[0177] <Control information> In each of the above embodiments, the enable flag has been described as an example of control information related to the present technology, but any other control information may be signaled.

[0178] <Surroundings / nearby> In this specification, positional relationships such as "nearby" and "around" may include not only spatial positional relationships but also temporal positional relationships.

[0179] <Computer> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, etc., capable of executing various functions by installing various programs.

[0180] FIG. 24 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0181] In a computer 900 shown in FIG. 24, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0182] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0183] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0184] In the computer configured as above, the series of processes described above are performed by the CPU 901, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data and the like necessary for the CPU 901 to execute various processes as appropriate.

[0185] The program executed by the computer can be applied by recording it on a removable medium 921 such as a package medium, for example. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by mounting the removable medium 921 on the drive 915.

[0186] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0187] In addition, this program can also be installed in advance in the ROM 902 or the storage unit 913.

[0188] <Target of application of this technology> Although the above describes the application of the present technology to the encoding and decoding of point cloud data, the present technology is not limited to these examples and can be applied to the encoding and decoding of 3D data of any standard. For example, in encoding and decoding of mesh data, mesh data may be converted into point cloud data and encoded and decoded by applying the present technology. In other words, various processes such as encoding and decoding methods and specifications of various data such as 3D data and metadata are arbitrary as long as they do not conflict with the present technology described above. In addition, some of the processes and specifications described above may be omitted as long as they do not conflict with the present technology.

[0189] Furthermore, although the encoding device 200 and the decoding device 300 have been described above as application examples of the present technology, the present technology can be applied to any configuration.

[0190] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) in satellite broadcasting, wired broadcasting such as cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media.

[0191] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0192] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and cooperate with each other to perform processing via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (video images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0193] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.

[0194] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications are also arbitrary.

[0195] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Thus, the value that this "flag" can take may be, for example, a binary value of 1 / 0, or may be three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, it is assumed that identification information (including flags) is not only included in the bit stream, but also includes difference information of the identification information with respect to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information with respect to the reference information.

[0196] Also, various information (metadata, etc.) related to the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, data associated with each other may be combined into one piece of data, or each piece of data may be made into separate data. For example, information associated with coded data (image) may be transmitted on a transmission path different from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium different from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in any unit, such as multiple frames, one frame, or a part of a frame.

[0197] In this specification, the terms "combine," "multiplex," "append," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0198] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0199] For example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0200] Also, for example, the above-mentioned program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0201] Also, for example, each step of one flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0202] Also, for example, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel, or individually at a required timing such as when a call is made. In other words, the processing of each step may be executed in an order different from the order described above, so long as no contradiction occurs. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0203] Also, for example, a plurality of techniques related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any of a plurality of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in other embodiments. Also, a part or all of any of the above-mentioned present technologies can be implemented in combination with other technologies not described above.

[0204] The present technology can also be configured as follows. (1) A coding control unit that performs control to prohibit the combined use of scalable coding, which is a coding method that generates coded data that can be decoded in a scalable manner, and scaling coding, which is a coding method that involves changing the tree structure of geometry data, in coding of a point cloud that represents a three-dimensional object as a collection of points. An information processing device comprising: (2) The scalable coding is lifting scalability, which encodes attribute data by lifting the attribute data using a reference structure similar to the tree structure of the geometry data. An information processing device as described in (1). (3) The scaling and encoding is geometry scaling, which scales and encodes the geometry data. An information processing device according to (1) or (2). (4) When the scalable coding is applied, the coding control unit controls so as to prohibit application of the scaling coding. An information processing device according to any one of (1) to (3). (5) The encoding control unit controls signaling between a scalable encoding enable flag, which is flag information regarding application of the scalable encoding, and a scaling encoding enable flag, which is flag information regarding application of the scaling encoding. (4) An information processing device. (6) When the encoding control unit signals the scalable encoding enable flag having a value indicating application of the scalable encoding, the encoding control unit controls the signaling of the scaling encoding enable flag having a value indicating non-application of the scaling encoding. (5) An information processing device according to the present invention. (7) When the scalable coding is applied, the coding control unit controls to signal, in a profile, the scalable coding enable flag having a value indicating application of the scalable coding and the scaling coding enable flag having a value indicating non-application of the scaling coding. An information processing device according to (5) or (6). (8) When the encoding control unit signals the scalable encoding enable flag having a value indicating application of the scalable encoding, the encoding control unit performs control so as to omit signaling of the scalable encoding enable flag. An information processing device according to any one of (5) to (7). (9) When the scaling coding is applied, the coding control unit controls so as to prohibit application of the scalable coding. An information processing device according to any one of (1) to (3). (10) The encoding control unit controls signaling of a scaling encoding enable flag that is flag information regarding application of the scaling encoding and a scalable encoding enable flag that is flag information regarding application of the scalable encoding. An information processing device according to (9). (11) When the encoding control unit signals the scaling encoding enable flag with a value indicating application of the scaling encoding, the encoding control unit controls to signal the scalable encoding enable flag with a value indicating non-application of the scalable encoding. An information processing device according to (10). (12) When the scaling coding is applied, the coding control unit controls to signal, in a profile, the scaling coding enable flag having a value indicating application of the scaling coding and the scalable coding enable flag having a value indicating non-application of the scalable coding. An information processing device according to (10) or (11). (13) When the encoding control unit signals the scaling encoding enable flag having a value indicating application of the scaling encoding, the encoding control unit performs control so as to omit signaling of the scalable encoding enable flag. An information processing device according to any one of (10) to (12). (14) a geometry data encoding unit that encodes the geometry data of the point cloud under control of the encoding control unit to generate encoded data of the geometry data; an attribute data encoding unit that encodes attribute data of the point cloud under control of the encoding control unit and generates encoded data of the attribute data; The information processing device according to any one of (1) to (13), further comprising: (15) The geometry data encoding unit a selection unit that selects whether to apply scaling to the geometry data under control of the encoding control unit; a geometry scaling unit that scales and merges the geometry data when application of the scaling of the geometry data is selected by the selection unit; an encoding unit that, when the selection unit has selected application of scaling to the geometry data, encodes the geometry data that has been scaled and merged by the geometry scaling unit, and, when the selection unit has selected not to apply scaling to the geometry data, encodes the geometry data that has not been scaled and merged; The information processing device according to (14) comprising: (16) The geometry data encoding unit A tree structure generating unit that generates a tree structure of the geometry data Further equipped with The geometry scaling unit performs the scaling and the merging to update the tree structure generated by the tree structure generation unit. An information processing device according to (15). (17) The attribute data encoding unit a selection unit that selects whether to apply scalable hierarchical layering to the attribute data by lifting the attribute data using a reference structure similar to a tree structure of the geometry data under control of the encoding control unit; a scalable hierarchical generation unit that performs the scalable hierarchical generation on the attribute data when application of the scalable hierarchical generation is selected by the selection unit; an encoding unit that encodes the attribute data that has been scalably layered by the scalable layering unit when application of the scalable layering is selected by the selection unit, and encodes the attribute data that has not been scalably layered when non-application of the scalable layering is selected by the selection unit; An information processing device according to any one of (14) to (16). (18) a geometry data decoding unit that decodes the encoded data of the geometry data generated by the geometry data encoding unit to generate the geometry data; a recolor processing unit that performs a recolor process on the attribute data by using the geometry data generated by the geometry data decoding unit; Further equipped with The attribute data encoding unit encodes the attribute data that has been recolored by the recolor processing unit. An information processing device according to any one of (14) to (17). (19) A bit stream generating unit that generates a bit stream including the coded data of the geometry data generated by the geometry data coding unit and the coded data of the attribute data generated by the attribute data coding unit. The information processing device according to any one of (14) to (18), further comprising: (20) In encoding a point cloud that represents a three-dimensional object as a set of points, a control is provided to prohibit the combined use of scalable encoding, which is an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, which is an encoding method that involves changing the tree structure of geometry data. Information processing methods. [Explanation of symbols]

[0205] 200 encoding device, 201 encoding control unit, 211 geometry data encoding unit, 212 geometry data decoding unit, 213 point cloud generation unit, 214 attribute data encoding unit, 215 bit stream generation unit, 231 voxel generation unit, 232 tree structure generation unit, 233 selection unit, 234 geometry scaling unit, 235 encoding unit, 251 selection unit, 252 scalable hierarchical processing unit, 253 hierarchical processing unit, 254 quantization unit, 255 encoding unit, 300 decoding device, 311 encoded data extraction unit, 312 geometry data decoding unit, 313 attribute data decoding unit, 314 point cloud generation unit

Claims

1. A coding control unit that controls to prohibit the combined use of scalable coding, which is a coding method that generates coded data that can be decoded in a scalable manner, and scaling coding, which is a coding method that involves changing the tree structure of geometry data, in coding of a point cloud that represents a three-dimensional object as a set of points. An information processing device comprising:

2. The scalable coding is lifting scalability, which encodes attribute data by lifting the attribute data using a reference structure similar to the tree structure of the geometry data. The information processing device according to claim 1 .

3. The scaling and encoding is geometry scaling in which the geometry data is scaled and encoded. The information processing device according to claim 1 .

4. The encoding control unit performs control so as to prohibit application of the scaling encoding when the scalable encoding is applied. The information processing device according to claim 1 .

5. The encoding control unit controls signaling between a scalable encoding enable flag, which is flag information regarding application of the scalable encoding, and a scaling encoding enable flag, which is flag information regarding application of the scaling encoding. The information processing device according to claim 4.

6. When the encoding control unit signals the scalable encoding enable flag having a value indicating application of the scalable encoding, the encoding control unit controls to signal the scaling encoding enable flag having a value indicating non-application of the scaling encoding. The information processing device according to claim 5 .

7. When the scalable coding is applied, the coding control unit controls to signal, in a profile, the scalable coding enable flag having a value indicating application of the scalable coding and the scaling coding enable flag having a value indicating non-application of the scaling coding. The information processing device according to claim 5 .

8. The encoding control unit performs control so as to omit signaling of the scalable encoding enable flag when signaling the scalable encoding enable flag having a value indicating application of the scalable encoding. The information processing device according to claim 5 .

9. The encoding control unit performs control so as to prohibit application of the scalable encoding when the scaling encoding is applied. The information processing device according to claim 1 .

10. The encoding control unit controls signaling between a scaling encoding enable flag, which is flag information regarding application of the scaling encoding, and a scalable encoding enable flag, which is flag information regarding application of the scalable encoding. The information processing device according to claim 9.

11. When the encoding control unit signals the scaling encoding enable flag with a value indicating application of the scaling encoding, the encoding control unit controls to signal the scalable encoding enable flag with a value indicating non-application of the scalable encoding. The information processing device according to claim 10.

12. When the scaling coding is applied, the coding control unit performs control so as to signal, in a profile, the scaling coding enable flag having a value indicating application of the scaling coding and the scalable coding enable flag having a value indicating non-application of the scalable coding. The information processing device according to claim 10.

13. The encoding control unit performs control so as to omit signaling of the scalable encoding enable flag when signaling the scaling encoding enable flag having a value indicating application of the scaling encoding. The information processing device according to claim 10.

14. a geometry data encoding unit that encodes the geometry data of the point cloud under control of the encoding control unit and generates encoded data of the geometry data; an attribute data encoding unit that encodes attribute data of the point cloud under control of the encoding control unit and generates encoded data of the attribute data; The information processing device according to claim 1 , further comprising:

15. The geometry data encoding unit includes: a selection unit that selects whether to apply scaling to the geometry data under control of the encoding control unit; a geometry scaling unit that scales and merges the geometry data when application of the scaling of the geometry data is selected by the selection unit; an encoding unit that, when the selection unit has selected application of scaling to the geometry data, encodes the geometry data that has been scaled and merged by the geometry scaling unit, and, when the selection unit has selected not to apply scaling to the geometry data, encodes the geometry data that has not been scaled and merged; The information processing device according to claim 14 .

16. The geometry data encoding unit includes: a tree structure generating unit for generating a tree structure of the geometry data; Further equipped with The geometry scaling unit updates the tree structure generated by the tree structure generation unit by performing the scaling and the merging. The information processing device according to claim 15.

17. The attribute data encoding unit includes: a selection unit that selects whether to apply scalable hierarchical layering to the attribute data by lifting the attribute data using a reference structure similar to a tree structure of the geometry data under control of the encoding control unit; a scalable hierarchical generation unit that performs the scalable hierarchical generation on the attribute data when application of the scalable hierarchical generation is selected by the selection unit; an encoding unit that encodes the attribute data that has been scalably layered by the scalable layering unit when application of the scalable layering is selected by the selection unit, and encodes the attribute data that has not been scalably layered when non-application of the scalable layering is selected by the selection unit; The information processing device according to claim 14.

18. a geometry data decoding unit that decodes the encoded data of the geometry data generated by the geometry data encoding unit and generates the geometry data; a recolor processing unit that performs a recolor process on the attribute data by using the geometry data generated by the geometry data decoding unit; Further equipped with The attribute data encoding unit encodes the attribute data that has been recolored by the recolor processing unit. The information processing device according to claim 14.

19. a bit stream generating unit that generates a bit stream including the encoded data of the geometry data generated by the geometry data encoding unit and the encoded data of the attribute data generated by the attribute data encoding unit; The information processing device according to claim 14 , further comprising:

20. An information processing device comprising: In encoding a point cloud that represents a three-dimensional object as a set of points, a control is performed to prohibit the combined use of scalable encoding, which is an encoding method that generates encoded data that can be decoded in a scalable manner, and scaling encoding, which is an encoding method that involves changing the tree structure of geometry data. Information processing methods.

Images