Information processing device and method

Abstract

The present disclosure relates to an information processing apparatus and method that enables scalable decoding of attribute information to be performed more easily. For attribute information of a point cloud representing a three-dimensional object, a process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point where the attribute information is referenced when deriving the prediction value, is recursively repeated for the reference point to perform hierarchy of the attribute information. During hierarchy, the prediction point is selected such that a point exists in a voxel of one upper layer to which the voxel containing the point of the current layer belongs. The present disclosure may be applied, for example, to an information processing apparatus, an image processing apparatus, an encoding apparatus, a decoding apparatus, an electronic device, an information processing method, or a program.

Description

Technology Field

[0001] The present disclosure relates to an information processing apparatus and method, and in particular to an information processing apparatus and method that enables scalable decoding of attribute information to be performed more easily. Background Technology

[0002] Conventionally, a method for encoding 3D data representing a three-dimensional structure, such as a point cloud, has been considered (see, for example, Non-patent Literature 1). The data of a point cloud consists of location information and attribute information of each point. Therefore, encoding of the point cloud is performed for each of the location information and attribute information. As a method for encoding attribute information, it has been proposed to use a technique called Lifting (see, for example, Non-patent Literature 2).

[0003] However, scalable decoding of such point cloud encoded data was considered. For example, when representing a vast 3D space with a point cloud, the amount of encoded data becomes very large. Decoding the entire amount would unnecessarily increase the load, so it was required to decode scalably so that the data of the necessary objects could be obtained at the required resolution. Prior art literature

[0004] R. Mekuria, Student Member IEEE, K. Blom, P. Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video", tcsvt_paper_submitted_february.pdfKhaled Mammou, Alexis Tourapis, Jungsun Kim, Fabrice Robinet, Valery Valentin, Yeping Su, "Lifting Scheme for Lossy Attribute Encoding in TMC1", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42640, April 2018, San Diego, US The problem to be solved

[0005] However, in the case of conventional encoding methods, the highest resolution positional information was required for decoding encoded data of attribute information. Consequently, positional information had to be decoded at unnecessarily high resolution, raising concerns that the load of the decoding process would increase.

[0006] The present disclosure is made in consideration of such circumstances and is intended to enable scalable decoding of attribute information to be performed more easily. means of solving the problem

[0007] An information processing device of one aspect of the present technology comprises a layering unit that performs layering of attribute information by recursively repeating a process for the reference points, wherein, for attribute information of a point cloud representing a three-dimensional shape object, the process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value. The layering unit is an information processing device that, during layering, selects the prediction points such that the points also exist in the voxels of one upper layer to which the voxels of the current layer that exist exist.

[0008] An information processing method of one aspect of the present technology is an information processing method that, for attribute information of a point cloud representing a three-dimensional shape object, performs hierarchy of attribute information by recursively repeating a process for the reference points to classify each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value, and selects the prediction point such that, during the hierarchy, the point exists in the voxel of the upper layer to which the voxel containing the point of the current layer belongs.

[0009] An information processing device of another aspect of the present technology comprises an inverse layering unit that performs inverse layering of the layered attribute information by recursively repeating, for the reference points, a process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value, with respect to the attribute information of a point cloud representing a three-dimensional shape object. The inverse layering unit, when performing the inverse layering, derives the prediction value of the attribute information of the prediction point using the attribute information of the reference point and the position information of the resolution of the layer other than the lowest layer of the point cloud in each layer, and derives the attribute information of the prediction point using the prediction value and the difference value.

[0010] Another aspect of the information processing method of the present invention is an information processing method that, for attribute information of a point cloud representing a three-dimensional shape object, performs inverse layering of the layered attribute information by recursively repeating a process for the reference points to classify each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point where the attribute information is referenced when deriving the prediction value; and, during the inverse layering, derives the prediction value of the attribute information of the prediction point using the attribute information of the reference point and the position information of the resolution of the layer other than the lowest layer of the point cloud in each layer, and derives the attribute information of the prediction point using the prediction value and the difference value.

[0011] Another aspect of the information processing device of the present invention is an information processing device comprising a generating unit that generates information representing a hierarchical structure of location information of a point cloud representing a three-dimensional shape object, and a layering unit that performs layering of attribute information of the point cloud to correspond to the hierarchical structure of the location information based on the information generated by the generating unit.

[0012] Another aspect of the information processing device of the present technology is an information processing device comprising a generating unit that generates information representing a hierarchical structure of location information of a point cloud representing a three-dimensional shape object, and a reverse layering unit that performs reverse layering of attribute information based on the information generated by the generating unit, such that the hierarchical structure of attribute information of the point cloud corresponds to the hierarchical structure of the location information.

[0013] In an information processing apparatus and method of one aspect of the present technology, regarding attribute information of a point cloud representing a three-dimensional object, a process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value, is recursively repeated for the reference point to perform hierarchy of attribute information, and during the hierarchy, a prediction point is selected such that a point exists in a voxel of one upper layer to which a voxel to which a point of the current layer exists also belongs.

[0014] In an information processing apparatus and method of another aspect of the present technology, for attribute information of a point cloud representing a three-dimensional shape object, a process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value, is recursively repeated for the reference point, thereby performing inverse layering of the layered attribute information. During the inverse layering, at each layer, the prediction value of the attribute information of the prediction point is derived using the attribute information of the reference point and the position information of the resolution of the layer other than the lowest layer of the point cloud, and the attribute information of the prediction point is derived using the prediction value and the difference value.

[0015] In another aspect of the information processing device of the present technology, information representing the hierarchical structure of location information of a point cloud representing a three-dimensional object is generated, and based on the generated information, the attribute information of the point cloud is hierarchized to correspond to the hierarchical structure of the location information.

[0016] In an information processing device of another aspect of the present technology, information representing the hierarchical structure of location information of a point cloud representing a three-dimensional object is generated, and based on the generated information, the hierarchical structure of the attribute information of the point cloud corresponds to the hierarchical structure of the location information, and thus the inverse hierarchy of the attribute information is performed. Brief explanation of the drawing

[0017] Figure 1 is a diagram illustrating an example of the layering of location information. Figure 2 is a diagram illustrating an example of lifting. Figure 3 is a diagram illustrating an example of lifting. Figure 4 is a diagram illustrating an example of lifting. Figure 5 is a diagram illustrating an example of quantization. Figure 6 is a diagram illustrating an example of a method for hierarchizing attribute information. Figure 7 is a diagram illustrating an example of the hierarchy of attribute information. Figure 8 is a diagram illustrating an example of inverse hierarchy of attribute information. Figure 9 is a diagram illustrating an example of the hierarchy of attribute information. Figure 10 is a diagram illustrating an example of the hierarchy of attribute information. Figure 11 is a diagram illustrating an example of inverse hierarchy of attribute information. Figure 12 is a diagram illustrating an example of a quantization method. Figure 13 is a diagram illustrating an example of quantization. FIG. 14 is a block diagram illustrating a main configuration example of an encoding device. FIG. 15 is a block diagram illustrating a main configuration example of an attribute information encoding unit. FIG. 16 is a block diagram illustrating a main configuration example of a hierarchical processing unit. Figure 17 is a flowchart illustrating an example of the flow of encoding processing. Figure 18 is a flowchart illustrating an example of the flow of attribute information encoding processing. Figure 19 is a flowchart illustrating an example of the flow of hierarchical processing. Figure 20 is a flowchart illustrating an example of the flow of quantization processing. Figure 21 is a flowchart illustrating an example of the flow of hierarchical processing. Figure 22 is a flowchart illustrating an example of the flow of quantization processing. FIG. 23 is a block diagram illustrating a main configuration example of a decoding device. FIG. 24 is a block diagram illustrating a main configuration example of an attribute information decoding unit. FIG. 25 is a block diagram illustrating a main configuration example of a reverse hierarchical processing unit. Figure 26 is a flowchart illustrating an example of the flow of decoding processing. Figure 27 is a flowchart illustrating an example of the flow of attribute information decoding processing. Figure 28 is a flowchart illustrating an example of the flow of inverse quantization processing. Figure 29 is a flowchart illustrating an example of the flow of reverse hierarchical processing. Figure 30 is a flowchart illustrating an example of the flow of inverse quantization processing. Figure 31 is a diagram illustrating an example of an Octree of location information to which DCM is applied. FIG. 32 is a diagram illustrating an example of an Octree referenced in the processing of attribute information. Figure 33 is a diagram illustrating an example of an Octree in the encoding of attribute information. Figure 34 is a diagram illustrating an example of an Octree in decoding attribute information. FIG. 35 is a diagram illustrating an example of an Octree in the encoding of attribute information. Figure 36 is a diagram illustrating an example of an Octree in decoding attribute information. Figure 37 is a diagram illustrating an example of deriving the Quantization Weight. Figure 38 is a diagram illustrating an example of a hierarchical structure of location information. Figure 39 is a drawing illustrating an example of label attachment. Figure 40 is a diagram illustrating an example of a hierarchical structure of location information. Figure 41 is a drawing illustrating an example of label attachment. Figure 42 is a drawing illustrating an example of label attachment. FIG. 43 is a block diagram illustrating a main configuration example of an encoding device. Figure 44 is a flowchart illustrating an example of the flow of encoding processing. Figure 45 is a flowchart illustrating an example of the flow of hierarchical processing. Figure 46 is a flowchart illustrating an example of the flow of decoding processing. Figure 47 is a flowchart illustrating an example of the flow of reverse hierarchical processing. FIG. 48 is a block diagram illustrating a main configuration example of a computer. Specific details for implementing the invention

[0018] Hereinafter, embodiments for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. In addition, the description will be carried out in the following order.

[0019] 1. Scalable decoding

[0020] 2. First embodiment (encoding device)

[0021] 3. Second embodiment (decoding device)

[0022] 4. DCM

[0023] 5. Quantization Weight

[0024] 6. LoDGeneration

[0025] 7. Bookkeeping

[0026] <1. Scalable Decoding>

[0027] Literature supporting technical content and terminology, etc.

[0028] The scope disclosed in the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent literature that were known at the time of filing.

[0029] Non-patent literature 1: (described above)

[0030] Non-patent literature 2: (described above)

[0031] Non-patent Document 3: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "Advanced video coding for generic audiovisual services", H.264, 04 / 2017

[0032] 비특허 문헌 4:TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU(International Telecommunication Union), "High efficiency video coding", H.265, 12 / 2016

[0033] 비특허 문헌 5: Jianle Chen, Elena Alshina, Gary J. Sullivan, Jens-Rainer, Jill Boyce, "Algorithm Description of Joint Exploration Test Model 4", JVET-G1001_v1, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 7th Meeting: Torino, IT, 13-21 July 2017

[0034] 비특허 문헌 6: Sebastien Lasserre, David Flynn, "[PCC] Inference of a mode using point location direct coding in TMC3", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42239, January 2018, Gwangju, Korea

[0035] In other words, the contents described in the aforementioned non-patent literature also serve as a basis for determining the support requirements. For example, even if there is no direct description in the embodiments of the Quad-Tree Block Structure described in Non-Patent Literature 4 or the QTBT (Quad Tree Plus Binary Tree) Block Structure described in Non-Patent Literature 4, they are deemed to be within the scope of the disclosure of the present technology and satisfy the support requirements of the claims. Furthermore, likewise, for technical terms such as parsing, syntax, and semantics, even if there is no direct description in the embodiments, they are deemed to be within the scope of the disclosure of the present technology and satisfy the support requirements of the claims.

[0036] Point Cloud

[0037] Conventionally, 3D data such as point clouds, which represent a three-dimensional structure based on point location information or attribute information, and meshes, which are composed of vertices, edges, and faces and define a three-dimensional shape using polygonal representations, have existed.

[0038] For example, in the case of a point cloud, a three-dimensional structure (an object with a three-dimensional shape) is represented as a set of multiple points. That is, the data of a point cloud (also referred to as point cloud data) is composed of location information or attribute information of each point in the point cloud. Attribute information includes, for example, color information, reflectance information, and normal information. Therefore, while the data structure is relatively simple, any three-dimensional structure can be represented with sufficient precision by using a sufficiently large number of points.

[0039] <octree>

[0040] Since the amount of such point cloud data is relatively large, compression of the amount of data through encoding was considered, for example, as described in Non-Patent Literature 1. For example, first, the location information of the point cloud is encoded (decoded), and then the attribute information is encoded (decoded). During encoding, the location information is quantized using voxels and also hierarchized by Octree.

[0041] A voxel is a small area of ​​a predetermined size that divides a three-dimensional space. Position information is corrected so that points are placed for each of these voxels. To explain in two dimensions, for example, as shown in FIG. 1A, a certain area (10) is divided by voxels (11-1) represented as squares, and position information of each point (12-1) is corrected so that points (12-1) represented as black circles are placed for each of the voxels (11-1). That is, position information is quantized according to the size of these voxels. Also, in FIG. 1A, only one voxel (11-1) is labeled, but all squares within the area (10) of FIG. 1A are voxels (11-1). Likewise, all black circles shown in FIG. 1A are points (12-1).

[0042] In an octree, one voxel is divided into two (i.e., eight) in each of the x, y, and z directions to form a voxel of one lower layer (also called LoD). In other words, two voxels (i.e. eight voxels) arranged in each of the x, y, and z directions are combined to form a voxel of one upper layer (LoD). Then, in each layer, position information is quantized by the voxel.

[0043] For example, in the case of A in FIG. 1, since it is two-dimensional, four voxels (11-1) arranged vertically and horizontally as in B in FIG. 1 are integrated to form one upper voxel (11-2) indicated by a thick line. Then, position information is quantized using this voxel (11-2). That is, if a point (12-1) (A in FIG. 1) exists within the voxel (11-2), the position information is corrected so that the point (12-1) is converted into a point (12-2) corresponding to the voxel (11-2). Also, in B in FIG. 1, only one voxel (11-1) is assigned a symbol, but all squares indicated by dotted lines within the area (10) of B in FIG. 1 are voxels (11-1). In addition, in Fig. 1B, only one voxel (11-2) is labeled, but all squares indicated by thick lines within the area (10) of Fig. 1B are voxels (11-2). Likewise, all black circles shown in Fig. 1B are points (12-2).

[0044] Likewise, as in Fig. 1C, four voxels (11-2) arranged vertically and horizontally are combined to form one upper voxel (11-3) indicated by a thick line. Then, position information is quantized using this voxel (11-3). That is, if a point (12-2) (B in Fig. 1) exists within the voxel (11-3), the point (12-2) is converted into a point (12-3) corresponding to the voxel (11-3) by correcting the position information. Additionally, in Fig. 1C, only one voxel (11-2) is assigned a symbol, but all squares indicated by dotted lines within the area (10) of Fig. 1C are voxels (11-2). In addition, in Fig. 1C, only one voxel (11-3) is labeled, but all squares indicated by thick lines within the area (10) of Fig. 1C are voxels (11-3). Likewise, all black circles shown in Fig. 1C are points (12-3).

[0045] Likewise, as in D of FIG. 1, four voxels (11-3) arranged vertically and horizontally are combined to form one upper voxel (11-4) indicated by a thick line. Then, position information is quantized using this voxel (11-4). That is, if a point (12-3) (C of FIG. 1) exists within the voxel (11-4), the position information is corrected so that the point (12-3) is converted into a point (12-4) corresponding to the voxel (11-4). Also, in D of FIG. 1, only one voxel (11-3) is assigned a symbol, but all squares indicated by dotted lines within the area (10) of D of FIG. 1 are voxels (11-3).

[0046] By doing this, location information is layered in correspondence with the hierarchical structure of the voxels.

[0047] <lifting>

[0048] In contrast, when encoding attribute information, the position information is treated as known by including degradation caused by encoding, and encoding is performed using the positional relationship between points. As a method for encoding such attribute information, a method using RAHT (Region Adaptive Hierarchical Transform) or a transformation called Lifting as described in Non-Patent Document 2 has been considered. By applying these techniques, attribute information can be hierarchized like an Octree of position information.

[0049] For example, in the case of Lifting, the attribute information of each point is encoded as the difference value with the predicted value derived using the attribute information of other points. Then, the points that perform the derivation of that difference value (i.e., the derivation of the predicted value) are selected hierarchically.

[0050] For example, to explain in two dimensions, in the hierarchy shown in A of FIG. 2, among the points (P0 to P9) indicated by circles, the points (P7, P8, P9) indicated by white circles are selected as prediction points from which prediction values ​​are derived, and the other points (P0 to P6) are selected as reference points from which attribute information is referenced when deriving the prediction values. Then, the prediction value of the prediction point (P7) is derived by referencing the attribute information of the reference points (P0 and P1). The prediction value of the prediction point (P8) is derived by referencing the attribute information of the reference points (P2 and P3). The prediction value of the prediction point (P9) is derived by referencing the attribute information of the reference points (P4 to P6). That is, in this hierarchy, the difference values ​​of each point (P7 to P9) are derived.

[0051] In that one upper layer, as shown in Fig. 2B, for the points (P0 to P6) selected as reference points in the layer of Fig. 2A (one lower layer), the classification (distinction) of prediction points and reference points is performed in the same way as in the case of layer of Fig. 2A.

[0052] For example, in Fig. 2B, points (P1, P3, P6) indicated by gray circles are selected as prediction points, and points (P0, P2, P4, P5) indicated by black circles are selected as reference points. Then, the predicted value of the prediction point (P1) is derived by referring to the attribute information of the reference points (P0 and P2). The predicted value of the prediction point (P3) is derived by referring to the attribute information of the reference points (P2 and P4). The predicted value of the prediction point (P6) is derived by referring to the attribute information of the reference points (P4 and P5). That is, in this layer, the difference value of each of the points (P1, P3, P6) is derived.

[0053] In that one upper layer, as shown in Fig. 2C, the classification (division) of points (P0, P2, P4, P5) selected as reference points in the layer of Fig. 2B (one lower layer) is performed.

[0054] By recursively repeating this classification for reference points of one lower layer, attribute information is layered.

[0055] Classification of Points

[0056] The procedure for classifying (dividing) points in this Lifting process is explained in more detail. As described above, the classification of points in Lifting is performed in the order from the lower layer to the upper layer. In each layer, first, each point is sorted in the order of Morton code. Next, the leading point of the column of points arranged in the order of Morton code is selected as the reference point. Next, points located near the reference point (neighboring points) are searched, and the searched points (neighboring points) are set as prediction points (also called index points).

[0057] For example, as shown in FIG. 3, points are searched within a circle (22) of radius R centered on a reference point (21) of the processing target. This radius R is pre-set for each hierarchy. In the example of FIG. 3, points (23-1) to (23-4) are detected and set as predicted points.

[0058] Next, the same classification is performed on the remaining points. That is, among the points that are not currently selected as either reference points or prediction points, the leading point in Morton code order is selected as the reference point, and points near that reference point are searched and set as prediction points.

[0059] When the processing described above is repeated until all points are classified, the processing of that layer is terminated, and the processing target is transferred to one higher layer. Then, the procedure described above is repeated for that layer. That is, each point selected as a reference point in one lower layer is sorted in Morton code order and classified into reference points and prediction points as described above. By repeating the processing as above, attribute information is layered.

[0060] <Derivation of Predicted Values>

[0061] In addition, as described above, in the case of Lifting, the predicted value of the attribute information of the prediction point is derived using the attribute information of the reference point surrounding the prediction point. For example, as shown in FIG. 4, the predicted value of the prediction point Q(i,j) is derived by referring to the attribute information of reference points (P1) to P3. In this case, as shown in Equation (1) below, the attribute information of each reference point is derived by integrating and weighting it with a weight (α(P,Q(i,j))) based on the inverse of the distance between the prediction point and the reference point. Here, A(P) represents the attribute information of the point (P).

[0062]

[0063] The location information of the highest resolution (i.e., the lowest layer) was used to derive the distance between this prediction point and its reference point.

[0064] Quantization

[0065] In addition, attribute information is quantized and encoded after being layered as described above. During the quantization, the attribute information (difference value) of each point is weighted according to the layered structure, as shown in the example of FIG. 5. This weight (Quantization Weight) W is derived for each point using the weight of the lower layer, as shown in FIG. 5. In addition, this weight can also be used in Lifting (layering of attribute information) to improve compression efficiency.

[0066] Hierarchical structure discrepancy

[0067] As mentioned above, since the method of hierarchizing attribute information differs from that of the Octree for location information, it is not guaranteed that the hierarchical structure of attribute information matches the hierarchical structure of location information. Consequently, there was a concern that cases might arise where, at intermediate resolutions (layers higher than the lowest layer), the points of attribute information and the points of location information do not coincide, making it impossible to accurately predict attribute information even when using location information from those intermediate resolutions (i.e., inverse hierarchization of attribute information becomes difficult).

[0068] In other words, in order to accurately decode (re-layer) attribute information, it was necessary to decode location information down to the lowest layer regardless of the layer, which raised concerns about an increased load on the decoding process.

[0069] <Discrepancy in position>

[0070] Furthermore, as mentioned above, since position information is quantized by voxels, there is a possibility that the position information of a point may change depending on the hierarchy. For this reason, there was a concern that cases might occur where attribute information cannot be accurately predicted using position information at intermediate resolution (i.e., inverse hierarchy of attribute information is difficult).

[0071] In other words, in order to accurately decode (re-layer) attribute information, it was necessary to decode location information down to the lowest layer regardless of the layer, which raised concerns about an increased load on the decoding process.

[0072] <Hierarchy of Attribute Information: Method 1>

[0073] Thus, as indicated in the top row of the table shown in Fig. 6, a prediction point is selected so that attribute information remains in the voxel of the one upper layer (LoD) to which the voxel containing the point belongs (Method 1).

[0074] That is, regarding the attribute information of a point cloud representing a 3D object, the attribute information is stratified by recursively repeating the process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point where the attribute information is referenced when deriving the prediction value, and during the stratification, the prediction point is selected such that the point also exists in the voxel of the upper layer to which the voxel containing the point of the current layer belongs.

[0075] For example, in an information processing device, regarding attribute information of a point cloud representing a three-dimensional shape object, a layering unit is provided to perform layering of attribute information by recursively repeating a process for reference points to classify each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point where the attribute information is referenced when deriving the prediction value. The layering unit selects the prediction point such that, during layering, the point exists in the voxel of the upper layer to which the voxel in the current layer belongs.

[0076] To explain in two dimensions, for example, encoding is performed as shown in FIG. 7. For example, as shown in A of FIG. 7, position information is set so that points (102-1) to points (102-9) are arranged in a predetermined spatial area (100) according to a predetermined hierarchy of voxels (101-1). Also, when there is no need to distinguish between voxels (101-1) to voxels (101-3), they are referred to as voxels (101). Also, when there is no need to distinguish between points (102-1) to points (102-9), they are referred to as points (102).

[0077] In this layer, as shown in FIG. 7B, points (102-1) to (102-9) are classified into prediction points and reference points so that points also exist in the voxel (101-2) of the upper layer of the voxel (101-1) in which points (102-1) to (102-9) exist. In the example of FIG. 7B, points (102-3), (102-5), and (102-8) indicated by white circles are set as prediction points, and other points are set as reference points.

[0078] Likewise, in one upper layer, points (102) are classified into prediction points and reference points so that points exist in the voxel (101-3) of one upper layer of the voxel (101-2) in which points (102-1), (102-2), (102-4), (102-6), (102-7), and (102-9) exist (Fig. 7C). In the example of Fig. 7C, points (102-1), (102-4), and (102-7) indicated by gray circles are set as prediction points, and other points are set as reference points.

[0079] By doing so, as shown in Fig. 7D, the voxels (101-3) in the lower layer where a point (102) exists are layered such that there is one point (102). This processing is performed for each layer. That is, layering similar to that of an Octree can be realized.

[0080] Decoding is performed in the reverse order of FIG. 7, for example, as shown in FIG. 8. For example, as shown in A of FIG. 8, position information is set so that points (102-2), points (102-6), and points (102-9) are arranged in a predetermined spatial area (100) according to a predetermined hierarchy of voxels (101-3) (a state similar to D of FIG. 7).

[0081] In this one lower layer, as shown in Fig. 8B, using the attribute information of points (102-2), points (102-6), and points (102-9) per voxel (101-3), the predicted values ​​of points (102-1), points (102-4), and points (102-7) are derived and added to the difference values, and the attribute information of points (102) per voxel (101-2) is restored (a state similar to that of Fig. 7C).

[0082] In addition, for one lower layer, as shown in Fig. 8C, using the attribute information of points (102-1), (102-2), (102-4), (102-6), (102-7), and (102-9) per voxel (101-2), the predicted values ​​of points (102-3), (102-5), and (102-8) are derived, added to the difference values, and the attribute information is restored (the same state as in Fig. 7B).

[0083] By doing so, the attribute information of the point (102) per voxel (101-1) is restored (same state as A in Fig. 7), as shown in D of Fig. 8. That is, just like in the case of an Octree, the attribute information of each layer can be restored using the attribute information of the upper layer.

[0084] By doing so, the hierarchical structure of attribute information can be mapped to the hierarchical structure of location information. Therefore, since location information corresponding to each attribute information is obtained even at intermediate resolutions, the attribute information can be accurately decoded by decoding the location information and attribute information at those intermediate resolutions. Consequently, scalable decoding of attribute information can be performed more easily.

[0085] <Hierarchy of Attribute Information: Method 1-1>

[0086] When deriving the predicted value when applying the above-described method 1, the predicted value may be derived by using the position information of the resolution of the current layer (LoD) in both the encoding and decoding stages, as described in the second row from the top of the table shown in FIG. 6 (Method 1-1).

[0087] For example, regarding attribute information of a point cloud representing a three-dimensional object, the inverse layering of the layered attribute information may be performed by recursively repeating the process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point where the attribute information is referenced when deriving the prediction value, and during the inverse layering, the prediction value of the attribute information of the prediction point may be derived using the attribute information of the reference point and the position information of the resolution of the layer other than the lowest layer of the point cloud in each layer, and the attribute information of the prediction point may be derived using the prediction value and the difference value.

[0088] In addition, for example, in an information processing device, for attribute information of a point cloud representing a three-dimensional shape object, a reverse layering unit is provided to perform reverse layering of layered attribute information by recursively repeating a process for reference points to classify each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value. The reverse layering unit may, during the reverse layering, derive the prediction value of the attribute information of the prediction point using the attribute information of the reference point and the position information of the resolution of the layer other than the lowest layer of the point cloud in each layer, and derive the attribute information of the prediction point using the prediction value and the difference value.

[0089] In addition, for example, in reverse layering, the predicted value may be derived by using the attribute information of the reference point and the location information of the resolution of the current layer of the reference point and the prediction point.

[0090] As explained with reference to Fig. 4, the derivation of the predicted value of the prediction point is performed using attribute information of the reference point that is weighted according to the distance between the prediction point and the reference point.

[0091] The derivation of this distance may be performed using the positional information of the resolution of the current layer (current layer) being processed, in either the encoding or decoding process. That is, in the layering process, the predicted value of the prediction point may be derived by using the attribute information of a reference point weighted according to the distance based on the positional information of the resolution of the current layer of the point cloud.

[0092] In the case of encoding, in Fig. 7B, the distance between the prediction point and the reference point is derived using the resolution location information of the voxel (101-1), and in Fig. 7C, the distance between the prediction point and the reference point is derived using the resolution location information of the voxel (101-2). Likewise, in the case of decoding, in Fig. 8B, the distance between the prediction point and the reference point is derived using the resolution location information of the voxel (101-2), and in Fig. 8C, the distance between the prediction point and the reference point is derived using the resolution location information of the voxel (101-1).

[0093] By doing so, the same distance can be derived in both the encoding and decoding phases. That is, the same predicted value can be derived. Therefore, even when decoding attribute information at an intermediate resolution, the reduction in precision due to the error in distance calculation (the increase in error due to decoding) can be suppressed. More specifically, when decoding attribute information at an intermediate resolution, decoding can be performed more accurately than in the case of Method 1-1' described below. In addition, the increase in the load of encoding and decoding can be suppressed compared to the case of Method 1-2 described below.

[0094] In addition, in the case of decoding (inverse layering) (Fig. 8), the correspondence between the location information and attribute information of each point at the intermediate resolution is unclear. For example, in Fig. 8B, it is unclear which voxel (101-2) each point (102) is located in. Therefore, just as in the case of layering, by using location information to layer attribute information (layering up to the intermediate resolution to be decoded), the attribute information and location information of each point at each layer can be matched. By doing so, inverse layering can be performed more accurately.

[0095] <Hierarchy of Attribute Information: Method 1-1>

[0096] In addition, when deriving the predicted value when applying the above-described Method 1, the predicted value may be derived using the location information of the current layer (LoD) resolution in the decoding process, as described in the third row from the top of the table shown in FIG. 6 (Method 1-1'). That is, in the encoding process, since the location information of the maximum resolution (the resolution of the lowest layer) is known, the predicted value (more specifically, the distance between the reference point and the prediction point used to derive the predicted value) may be derived using the location information of the maximum resolution.

[0097] In other words, the predicted value of the prediction point may be derived by using attribute information of a reference point weighted according to distance based on the location information of the resolution of the lowest layer of the point cloud.

[0098] For example, if the layer shown in A of FIG. 9 (the same layer as in A of FIG. 7) is called the lowest layer, the derivation of predicted values ​​for points (102-3), points (102-5), and points (102-8) is performed using the positional information of the resolution of the voxel (101-1), as shown in B of FIG. 9. This is the same as in the case of B of FIG. 7.

[0099] However, in this case, as shown in Fig. 9C, the derivation of predicted values ​​for points (102-1), points (102-4), and points (102-7) is performed using position information of the resolution of voxel (101-1), just as in the case of Fig. 9B. That is, as shown in Fig. 9D, the position information of points (102-2), points (102-6), and points (102-9) per voxel (101-3) also has the resolution of voxel (101-1).

[0100] By doing so, the prediction accuracy can be improved compared to the case of Method 1-1, and thus the encoding efficiency can be improved compared to the case of Method 1-1. However, during decoding, as shown in D of FIG. 9, the location information of points (102-2), points (102-6), and points (102-9) per voxel (101-3) cannot have the resolution of voxel (101-1), so decoding is performed in the same way as in the case of FIG. 8 (Method 1-1). That is, the distance between the reference point and the prediction point is derived by using location information of different resolutions in encoding and decoding.

[0101] <Hierarchy of Attribute Information: Method 1-2>

[0102] In addition, when deriving the predicted value when applying the above-described Method 1, the predicted value may be derived using a virtual reference point as described in the bottom row of the table shown in FIG. 6 (Method 1-2). That is, in the layering, all points of the current layer may be classified as predicted points, a reference point may be set within a voxel of one upper layer, and the predicted value of each predicted point may be derived using the attribute information of the reference point.

[0103] In addition, for example, in reverse layering, the predicted value of each prediction point may be derived by using the attribute information of the reference point, the location information of the resolution of one upper layer of the current layer of the reference point, and the location information of the resolution of the current layer of the prediction point.

[0104] To explain in two dimensions, for example, encoding is performed as shown in FIG. 10. FIG. 10A illustrates the same hierarchy as FIG. 7, and in the spatial domain (100), points (111-1) to points (111-9) exist for each voxel (101-1), similar to points (102-1) to points (102-9). Also, when there is no need to explain points (111-1) to points (111-9) separately from each other, they are referred to as points (111).

[0105] In this case, unlike in FIG. 7, points (111-1) to (111-9) are all set as prediction points. And, as shown in FIG. 10B, virtual points (112-1) to (112-7) are set as reference points for each voxel (101-2) in the voxel (101-2) to which the voxel (101-1) belongs, which is one level above the voxel (101-1) and where their prediction points (points (111)) exist. Also, when there is no need to explain points (112-1) to (112-7) separately from each other, they are referred to as points (112).

[0106] The attribute information of the newly set virtual point (112) is derived by using the attribute information of the nearby point (111) through recoloring. Additionally, the location of the point (112) is arbitrary as long as it is within the voxel (101-2) and may be other than the center shown in Fig. 10 B.

[0107] As shown in Fig. 10B, the predicted value of each prediction point (point (111)) is derived using these reference points (point (112)).

[0108] And, in the one-level higher layer, as illustrated in Fig. 10C, the point (112) set in Fig. 10B is set as a prediction point. And, similar to the case of Fig. 10B, in the one-level higher layer of the voxel (101-2), virtual points (113-1) to (113-3) for each voxel (101-3) are set as reference points in the voxel (101-3) to which the voxel (101-2) to which their prediction point (point (112)) exists exists. Also, when there is no need to explain points (113-1) to (113-3) separately from each other, they are referred to as point (113).

[0109] The attribute information of the newly set virtual point (113) is derived by recolor processing, just as in the case of the point (112). As shown in Fig. 10C, the predicted value of each predicted point (point (112)) is derived using these reference points (point (113)).

[0110] By doing so, as illustrated in Fig. 10 D, the voxel (101-3) in the lower layer where a point (111) exists is layered such that there is one point (113). This processing is performed for each layer. That is, layering similar to that of an Octree can be realized.

[0111] Decoding is performed in the reverse order of FIG. 10, for example, as shown in FIG. 11. For example, as shown in A of FIG. 11, position information is set so that points (113-1) to points (113-3) are arranged according to voxels (101-3) of a predetermined hierarchy in a predetermined spatial area (100) (a state similar to D of FIG. 10).

[0112] In this one lower layer, as shown in B of FIG. 11, the attribute information of points (113-1) to points (113-3) per voxel (101-3) is used to derive the predicted values ​​of points (112-1) to points (112-7), which are added to the difference values, and the attribute information of points (112) per voxel (101-2) is restored (a state similar to C of FIG. 10).

[0113] In addition, in one lower layer, as shown in Fig. 11C, the attribute information of points (112-1) to (112-7) per voxel (101-2) is used to derive the predicted values ​​of points (111-1) to (111-9), add them to the difference values, and restore the attribute information (same state as in Fig. 10B).

[0114] By doing so, the attribute information of the point (111) per voxel (101-1) is restored (same state as A in Fig. 10), as shown in D of Fig. 11. That is, just like in the case of an Octree, the attribute information of each layer can be restored using the attribute information of the upper layer.

[0115] In addition, in this case, since prediction is performed using recolored virtual points, the prediction accuracy can be improved compared to the case of Method 1-1 or Method 1-1'. That is, the encoding efficiency can be improved compared to the case of Method 1-1 or Method 1-1'. Furthermore, in this method, since the distance between the reference point and the prediction point can be derived using position information of the same resolution for both encoding and decoding, decoding can be performed more accurately than in the case of Method 1-1'. However, in this case, the load increases accordingly because virtual points are set or recolored processing is performed.

[0116] <Hierarchy of Attribute Information: Combination>

[0117] In addition, any of the methods described above may be selected and applied. In that case, the encoding side selects and applies a method, transmits information indicating which method was selected from the encoding side to the decoding side, and the decoding side applies the same method as the encoding side based on that information.

[0118] <Reference of lower layer when deriving weights>

[0119] However, as explained in <Quantization>, the quantization weight W used for the quantization or layering of attribute information (difference value) is derived point by point using the weights of the lower layer. Therefore, during decoding, the weights of the lower layer were required to inversely quantize the attribute information of intermediate resolution. In other words, it was difficult to perform scalable decoding. To put it another way, since all layers had to be decoded to decode the desired attribute information of intermediate resolution, there was a concern that the load of the decoding process would increase.

[0120] <Weights: Method 1>

[0121] Therefore, as indicated in the top row of the table shown in FIG. 12, instead of the point-by-point weights described above, the quantization and in-quantization of attribute information may be performed using layer-by-layer (LoD) weights (Method 1). That is, the difference values ​​of each point of each layer generated as described above may be quantized and in-quantized using layer-by-layer weights.

[0122] For example, as illustrated in FIG. 13, a quantization weight is set to C0 for a certain layer LoD (N), and the attribute information (difference value) of all points in that layer LoD (N) is quantized and inversely quantized using the weight C0. In addition, a weight is set to C1 for the layer one higher LoD (N-1), and the attribute information (difference value) of all points in that layer LoD (N-1) is quantized and inversely quantized using the weight C1. In addition, a weight is set to C2 for the layer one higher LoD (N-2), and the attribute information (difference value) of all points in that layer LoD (N-2) is quantized and inversely quantized using the weight C2.

[0123] Furthermore, these weights C0, C1, and C2 are set independently of each other. When using point-specific weights, it is necessary to consider how to assign weights between points within the same hierarchy; this not only complicates the processing but also raises concerns that the independence of hierarchy-specific weights may become difficult, for example, as information from lower layers may be required. In contrast, hierarchy-specific weights can be derived more easily and independently of each other.

[0124] By applying independent weights between these layers, quantization and inverse quantization that do not depend on lower layers become possible. In other words, as described above, since referencing information from lower layers is unnecessary, scalable decoding can be realized more easily.

[0125] In addition, the higher the layer, the larger the value may be (in the example of Fig. 13, C0≤C1≤C2). Generally, the higher the layer, the more important the layered attribute information is because it influences the attribute information of more points (since it is used directly or indirectly in deriving the predicted value). Therefore, by quantizing and inversely quantizing the attribute information (difference value) of the higher layer using a larger weight, the reduction in the precision of encoding and decoding can be suppressed.

[0126] <Weights: Method 1-1>

[0127] In addition, when applying Method 1, the function for deriving weights in encoding and decoding (weight derivation function) may be shared, as described in the second row from the top of the table shown in FIG. 12 (Method 1-1). That is, the same weight derivation function may be used in encoding and decoding to derive the aforementioned hierarchical weights. By doing so, the same weights can be derived in encoding and decoding, thereby suppressing the reduction in precision due to quantization and inverse quantization.

[0128] Furthermore, this weight derivation function may be any function. That is, the weights for each layer derived by this weight derivation function are arbitrary. For example, it may be a function where higher layers yield larger weights (a function that increases monotonically with the layer). Additionally, the rate of increase of the weights per layer may or may not be constant. Moreover, the range of weights derived by this weight derivation function is also arbitrary.

[0129] <Weights: Method 1-1-1>

[0130] In addition, in that case, a predetermined function may be shared in advance as described in the third row from the top of the table shown in FIG. 12 (Method 1-1-1). That is, in encoding and decoding, the above-described layer-specific weights may be derived using a predetermined identical weight derivation function. By doing so, the weight derivation function can be shared more easily in encoding and decoding.

[0131] <Weights: Method 1-1-2>

[0132] In addition, as described in the fourth row from the top of the table shown in FIG. 12, parameters defining the weight derivation function (e.g., information specifying the function or coefficients, etc.) may be shared (Method 1-1-2). That is, the weight derivation function may be defined during encoding, and the parameters defining the weight derivation function may be transmitted to the decoding side (included in the bit stream). That is, the quantized difference value and the parameters defining the function used when deriving the layer-by-layer weight may be encoded, and encoded data may be generated. Then, the encoded data may be decoded, and the quantized difference value and the parameters defining the function used when deriving the layer-by-layer weight may be obtained. By doing so, the weight derivation function used during encoding can be easily restored based on the parameters included in the bit stream, so the same weight derivation function as in the case of encoding can be easily applied during decoding.

[0133] <Weights: Method 1-2>

[0134] In addition, when applying Method 1, weights may be shared in encoding and decoding as described in the fifth row from the top of the table shown in FIG. 12 (Method 1-2). That is, quantization and dequantization may be performed using the same weights in encoding and decoding. By doing so, the reduction in precision due to quantization and dequantization can be suppressed. Furthermore, the magnitude of the weights for each layer is arbitrary.

[0135] <Weights: Method 1-2-1>

[0136] In addition, in that case, a predetermined weight may be shared in advance as described in the sixth row from the top of the table shown in FIG. 12 (Method 1-2-1). That is, in encoding and decoding, quantization and inverse quantization may be performed using the same predetermined weights. By doing so, the weights can be shared more easily in encoding and decoding.

[0137] <Weights: Method 1-2-2>

[0138] In addition, as described in the 7th row from the top of the table shown in FIG. 12, weights may be transmitted from the encoding side to the decoding side (Method 1-2-2). That is, weights may be set during encoding and transmitted to the decoding side (included in the bit stream). That is, quantized difference values ​​and layer-specific weights may be encoded to generate encoded data. Then, the encoded data may be decoded to obtain quantized difference values ​​and layer-specific weights. By doing so, the weights included in the bit stream can be applied to the inverse quantization during decoding. Therefore, the weights used for quantization during encoding can be more easily applied to the inverse quantization during decoding.

[0139] <Weights: Method 2>

[0140] In addition, point-specific weights may be applied during quantization and inverse quantization. For example, as described in the eighth row from the top of the table shown in FIG. 12, point-specific weights may be defined and applied during encoding (quantization), and the point-specific weights may be transmitted to the decoding side (included in the bit stream) (Method 2). By doing so, quantization and inverse quantization can be performed using appropriate weights according to the attribute information (difference value) of each point, thereby suppressing the reduction in encoding efficiency.

[0141] <Weights: Method 2-1>

[0142] When applying this method 2, the weights of the entire defined point may be transmitted (included in the bit stream) as described in the ninth row from the top of the table shown in FIG. 12 (Method 2-1). By doing so, the weights included in the bit stream can be applied as they are during decoding (inverse quantization), making it easier to perform inverse quantization.

[0143] <Weights: Method 2-2>

[0144] In addition, when applying Method 2, the difference value between the weight of each defined point and a predetermined value may be transmitted (included in the bit stream), as indicated in the bottom row of the table shown in FIG. 12 (Method 2-2). This predetermined value is arbitrary. This predetermined value may be a value different from each weight, or the difference value between the weights may be transmitted. By doing so, for example, compared to the case of Method 2-1, the increase in the amount of information regarding the weights can be suppressed and the reduction in coding efficiency can be suppressed.

[0145] <Weight: Combination>

[0146] In addition, any of the methods described above may be selected and applied. In that case, the encoding side selects and applies a method, transmits information indicating which method was selected from the encoding side to the decoding side, and the decoding side applies the same method as the encoding side based on that information.

[0147] <Application of Weights to Hierarchy>

[0148] This weight may be used for the layering of attribute information. For example, in the case of Lifting, the difference value derived at each layer is updated using this weight (Quantization Weight), and the attribute information of the point set as the reference point is updated using the updated difference value. By doing so, the reduction in compression efficiency is suppressed. When performing such update processing in the layering of attribute information, the layer-specific weights described above may be applied.

[0149] By doing this, reference to lower-level information is unnecessary, making it easier to realize scalable decoding.

[0150] <2. First Embodiment>

[0151] Encoding device

[0152] Next, a device applying the present technology described above in <1. Scalable Decoding> will be described. FIG. 14 is a block diagram illustrating an example of the configuration of an encoding device, which is an embodiment of an information processing device applying the present technology. The encoding device (200) shown in FIG. 14 is a device that encodes a point cloud (3D data). The encoding device (200) encodes the point cloud by applying the present technology described above in <1. Scalable Decoding>.

[0153] In addition, FIG. 14 illustrates the main parts, such as processing units and data flow, and it cannot be said that everything is shown in FIG. 14. That is, in the encoding device (200), there may be processing units that are not indicated as blocks in FIG. 14, or processing or data flow that is not indicated as arrows in FIG. 14.

[0154] As illustrated in FIG. 14, the encoding device (200) has a location information encoding unit (201), a location information decoding unit (202), a point cloud generation unit (203), an attribute information encoding unit (204), and a bit stream generation unit (205).

[0155] The position information encoding unit (201) encodes the position information of the point cloud (3D data) input to the encoding device (200). This encoding method is optional. For example, processing such as filtering or quantization for noise suppression (de-noising) may be performed. The position information encoding unit (201) supplies the generated position information encoded data to the position information decoding unit (202) and the bit stream generation unit (205).

[0156] The location information decoding unit (202) acquires encoded data of location information supplied from the location information encoding unit (201) and decodes the encoded data. The decoding method may be any method corresponding to the encoding by the location information encoding unit (201). For example, processing such as filtering for denoising or inverse quantization may be performed. The location information decoding unit (202) supplies the generated location information (decoding result) to the point cloud generation unit (203).

[0157] The point cloud generation unit (203) acquires attribute information of a point cloud input to the encoding device (200) and location information (decoding result) supplied from the location information decoding unit (202). The point cloud generation unit (203) performs processing (recolor processing) to match the attribute information to the location information (decoding result). The point cloud generation unit (203) supplies the attribute information corresponding to the location information (decoding result) to the attribute information encoding unit (204).

[0158] The attribute information encoding unit (204) acquires location information (decoding result) and attribute information supplied from the point cloud generation unit (203). The attribute information encoding unit (204) uses the location information (decoding result) to encode the attribute information by the encoding method applying the present technology described above in <1. Scalable Decoding> and generates encoded data of the attribute information. The attribute information encoding unit (204) supplies the generated encoded data of the attribute information to the bit stream generation unit (205).

[0159] The bit stream generation unit (205) acquires encoded data of position information supplied from the position information encoding unit (201). Additionally, the bit stream generation unit (205) acquires encoded data of attribute information supplied from the attribute information encoding unit (204). The bit stream generation unit (205) generates a bit stream containing these encoded data. The bit stream generation unit (205) outputs the generated bit stream to the outside of the encoding device (200).

[0160] By configuring it in this way, the encoding device (200) can correspond the hierarchical structure of attribute information to the hierarchical structure of position information. In addition, by doing so, quantization and inverse quantization that do not depend on the lower layer become possible. That is, the encoding device (200) can make scalable decoding of attribute information easier.

[0161] In addition, these processing units (position information encoding unit (201) to bit stream generation unit (205)) may have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the processing described above. In addition, each processing unit may have, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and realize the processing described above by executing a program using them. Of course, each processing unit may have both configurations, realizing part of the processing described above by a logic circuit and the rest by executing a program. The configurations of each processing unit may be independent of each other, for example, some processing units may realize part of the processing described above by a logic circuit, other processing units may realize the processing described above by executing a program, and yet another processing unit may realize the processing described above by both a logic circuit and the execution of a program.

[0162] <Attribute Information Encoding Section>

[0163] FIG. 15 is a block diagram illustrating a main configuration example of an attribute information encoding unit (204) (Fig. 14). FIG. 15 illustrates the main parts, such as a processing unit and data flow, and FIG. 15 is not to say that everything is shown. That is, in the attribute information encoding unit (204), there may be a processing unit that is not indicated as a block in FIG. 15, or a processing or data flow that is not indicated as an arrow in FIG. 15.

[0164] As illustrated in FIG. 15, the attribute information encoding unit (204) has a hierarchical processing unit (211), a quantization unit (212), and an encoding unit (213).

[0165] The layering processing unit (211) performs processing regarding the layering of attribute information. For example, the layering processing unit (211) acquires attribute information or location information (decoding result) supplied to the attribute information encoding unit (204). The layering processing unit (211) layers the attribute information using the location information. At that time, the layering processing unit (211) performs layering by applying the technology described above in <1. Scalable Decoding>. That is, the layering processing unit (211) generates attribute information with a layered structure similar to the location information (corresponding the layered structure of the attribute information to the layered structure of the location information). The layering processing unit (211) supplies the layered attribute information (difference value) to the quantization unit (212).

[0166] The quantization unit (212) acquires attribute information (difference value) supplied from the hierarchical processing unit (211). The quantization unit (212) quantizes the attribute information (difference value). At that time, the quantization unit (212) performs quantization by applying the present technology described above in <1. Scalable Decoding>. That is, the quantization unit (212) quantizes the attribute information (difference value) using the hierarchical weights as described in <1. Scalable Decoding>. The quantization unit (212) supplies the quantized attribute information (difference value) to the encoding unit (213). Additionally, the quantization unit (212) supplies information regarding the weights applied to the quantization to the encoding unit (213) as needed.

[0167] The encoding unit (213) acquires quantized attribute information (difference value) supplied from the quantization unit (212). The encoding unit (213) encodes the quantized attribute information (difference value) and generates encoded data of the attribute information. This encoding method is optional. Additionally, if information regarding weights is supplied from the quantization unit (212), the encoding unit (213) includes that information (e.g., parameters defining a weight derivation function, weights, etc.) in the encoded data. The encoding unit (213) supplies the generated encoded data to the bit stream generation unit (205).

[0168] By performing hierarchy or quantization as described above, the attribute information encoding unit (204) can correspond the hierarchical structure of the attribute information to the hierarchical structure of the position information. In addition, quantization can be performed without relying on the lower layer. Furthermore, inverse quantization without relying on the lower layer becomes possible. That is, the attribute information encoding unit (204) can facilitate scalable decoding of the attribute information.

[0169] In addition, these processing units (hierarchical processing unit (211) to encoding unit (213)) may have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the processing described above. In addition, each processing unit may have, for example, a CPU, ROM, RAM, etc., and realize the processing described above by executing a program using them. Of course, each processing unit may have both configurations, realizing part of the processing described above by a logic circuit and the rest by executing a program. The configurations of each processing unit may be independent of each other, for example, some processing units may realize part of the processing described above by a logic circuit, other processing units may realize the processing described above by executing a program, and yet another processing unit may realize the processing described above by both a logic circuit and the execution of a program.

[0170] Hierarchy Processing Unit

[0171] FIG. 16 is a block diagram illustrating a main configuration example of a hierarchical processing unit (211) (Fig. 15). FIG. 16 illustrates the main parts, such as processing units and data flows, and FIG. 16 is not to say that everything is shown. That is, in the hierarchical processing unit (211), there may be processing units that are not indicated as blocks in FIG. 16, or processing or data flows that are not indicated as arrows, etc. in FIG. 16.

[0172] As illustrated in FIG. 16, the hierarchical processing unit (211) comprises a layered processing unit (221-1), a layered processing unit (221-2), … The layered processing unit (221-1) performs processing for the lowest layer (N) of the attribute information. The layered processing unit (221-2) performs processing for a layer (N-1) one layer above the lowest layer of the attribute information. When there is no need to explain these layered processing units (221-1), layered processing units (221-2), … separately, they are referred to as the layered processing unit (221). That is, the hierarchical processing unit (211) has layered processing units (221) corresponding to the number of layers of attribute information.

[0173] The layered processing unit (221) performs processing regarding the layering of attribute information of the current layer (current LoD) to be processed. For example, the layered processing unit (221) classifies each point of the current layer into a prediction point and a reference point (a prediction point of a higher layer) of that current layer, and derives a prediction value or a difference value. At that time, the layered processing unit (221) performs processing by applying the present technology described above in <1. Scalable Decoding>.

[0174] The hierarchical processing unit (221) has a point classification unit (231), a prediction unit (232), a calculation unit (233), an update unit (234), and a calculation unit (235).

[0175] The point classification unit (231) classifies each point in the current layer into a prediction point and a reference point. At that time, the point classification unit (231) selects a prediction point (in other words, a reference point) such that a point exists in a voxel of one upper layer to which a voxel containing a point in the current layer belongs, as described above in <1. Scalable Decoding>. That is, the point classification unit (231) classifies points by any method described with reference to the table in FIG. 6. The point classification unit (231) supplies attribute information H (N) of the point selected as the prediction point to the operation unit (233). Additionally, the point classification unit (231) supplies attribute information L (N) of the point selected as the reference point to the prediction unit (232) and the operation unit (235).

[0176] Additionally, the point classification unit (231), prediction unit (232), and update unit (234) share location information of each point.

[0177] The prediction unit (232) derives a predicted value P(N) of the attribute information of the predicted point by using attribute information L(N) or location information of the point selected as a reference point, supplied from the point classification unit (231). At that time, the prediction unit (232) derives the predicted value P(N) by applying the present technology described above in <1. Scalable Decoding>. The prediction unit (232) supplies the derived predicted value P(N) to the calculation unit (233).

[0178] The operation unit (233) subtracts the prediction value P(N) supplied by the prediction unit (232) from the attribute information H(N) supplied by the point classification unit (231) for each prediction point, and derives a difference value D(N). The operation unit (233) supplies the derived difference value D(N) of each prediction point to the quantization unit (212) (Fig. 15). Additionally, the operation unit (233) supplies the derived difference value D(N) of each prediction point to the update unit (234).

[0179] The update unit (234) obtains the difference value D(N) of the prediction point supplied from the operation unit (233). The update unit (234) updates the difference value D(N) by performing a predetermined operation and derives the coefficient U(N). This operation method is optional, but, for example, the operation as in Equation (2) below may be performed using the quantization weight W used for quantization.

[0180]

[0181] At that time, the update unit (234) independently derives the weight w used in this operation for each layer, as described above in <1. Scalable Decoding>. That is, the point classification unit (231) performs this operation using the weight w derived by applying any method described with reference to the table in FIG. 12. The update unit (234) supplies the derived coefficient U(N) to the operation unit (235).

[0182] The operation unit (235) acquires attribute information L(N) of a reference point supplied from the point classification unit (231). Additionally, the operation unit (235) acquires a coefficient U(N) supplied from the update unit (234). The operation unit (235) updates the attribute information L(N) by adding the coefficient U(N) to the attribute information L(N) (deriving the updated attribute information L'(N). By doing so, the reduction in compression efficiency can be suppressed. The operation unit (235) supplies the updated attribute information L'(N) to the point classification unit (231) of the hierarchical processing unit (221) which processes one upper layer (N-1). For example, in the case of FIG. 16, the operation unit (235) of the hierarchical processing unit (221-1) supplies the updated attribute information L'(N) to the hierarchical processing unit (221-2). The layered processing unit (221) supplied with the updated attribute information L'(N) performs the same processing with that point as the processing target (i.e., with one upper layer as the current layer).

[0183] By performing the layering as described above, the layering processing unit (211) can correspond the layered structure of the attribute information to the layered structure of the location information. In addition, layering can be performed using weights derived without relying on lower layers. That is, the layering processing unit (211) can make scalable decoding of the attribute information easier.

[0184] In addition, these processing units (point classification unit (231) to operation unit (235)) may have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the processing described above. In addition, each processing unit may have, for example, a CPU, ROM, RAM, etc., and realize the processing described above by executing a program using them. Of course, each processing unit may have both configurations, realizing part of the processing described above by a logic circuit and the rest by executing a program. The configurations of each processing unit may be independent of each other, for example, some processing units may realize part of the processing described above by a logic circuit, other processing units may realize the processing described above by executing a program, and yet another processing unit may realize the processing described above by both a logic circuit and the execution of a program.

[0185] <Flow of Encoding Processing>

[0186] Next, the processing performed by this encoding device (200) will be described. The encoding device (200) encodes the data of the point cloud by performing encoding processing. An example of the flow of this encoding processing will be explained with reference to the flowchart of FIG. 17.

[0187] When encoding processing is initiated, the position information encoding unit (201) of the encoding device (200) encodes the position information of the input point cloud in step S201 and generates encoded data of the position information.

[0188] In step S202, the position information decoding unit (202) decodes the encoded data of the position information generated in step S201 and generates position information.

[0189] In step S203, the point cloud generation unit (203) uses the attribute information of the input point cloud and the location information (decoding result) generated in step S202 to perform recolor processing and correspond the attribute information to the location information.

[0190] In step S204, the attribute information encoding unit (204) encodes the attribute information recolored in step S203 by executing attribute information encoding processing and generates encoded data of the attribute information. At that time, the attribute information encoding unit (204) performs processing by applying the present technology described above in <1. Scalable Decoding>. Details of the attribute information encoding processing will be described later.

[0191] In step S205, the bit stream generating unit (205) generates and outputs a bit stream including encoded data of position information generated in step S201 and encoded data of attribute information generated in step S204.

[0192] When the processing of step S205 is finished, the encoding process is terminated.

[0193] By performing the processing of each step in this manner, the encoding device (200) can correspond the hierarchical structure of attribute information to the hierarchical structure of position information. In addition, by doing so, quantization and inverse quantization that do not depend on the lower layer become possible. That is, the encoding device (200) can make scalable decoding of attribute information easier.

[0194] <Flow of Attribute Information Encoding Process>

[0195] Next, an example of the flow of the attribute information encoding process executed in step S204 of FIG. 17 will be explained with reference to the flowchart of FIG. 18.

[0196] When the attribute information encoding process is initiated, the layering processing unit (211) of the attribute information encoding unit (204) layers the attribute information by executing layering processing in step S221 and derives the difference value of the attribute information of each point. At that time, the layering processing unit (211) performs layering by applying the present technology described above in <1. Scalable Decoding>. Details of the layering processing will be described later.

[0197] In step S222, the quantization unit (212) quantizes each difference value derived in step S221 by performing quantization processing. At that time, the quantization unit (212) performs quantization by applying the present technology described above in <1. Scalable Decoding>. Details of the quantization processing will be described later.

[0198] In step S223, the encoding unit (213) encodes the quantized difference value in step S222 and generates encoded data of attribute information.

[0199] When the processing of step S223 is finished, the attribute information encoding process is finished, and the process returns to Fig. 17.

[0200] By processing each step in this manner, the attribute information encoding unit (204) can correspond the hierarchical structure of the attribute information to the hierarchical structure of the position information. In addition, quantization can be performed without relying on the lower layer. Furthermore, inverse quantization without relying on the lower layer becomes possible. That is, the attribute information encoding unit (204) can facilitate scalable decoding of the attribute information.

[0201] Flow of Hierarchical Processing

[0202] Next, an example of the flow of the layered processing executed in step S221 of FIG. 18 will be explained with reference to the flowchart of FIG. 19. Here, the case in which Method 1-1, described as the second from the top in the table of FIG. 6, is applied will be explained.

[0203] When layering processing is initiated, the layer-specific processing unit (221) of the layering processing unit (211) sets the current layer (current LoD) to be processed as the lowest layer (LoD) in step S241.

[0204] In step S242, the point classification unit (231) selects a predicted point such that a point exists in a voxel of one upper layer to which the voxel containing the current layer point belongs. In other words, the point classification unit (231) selects a reference point by this process.

[0205] For example, the point classification unit (231) sets one reference point for each voxel of the upper layer to which each voxel containing a current layer point belongs. In addition, if there are multiple candidate points, the point classification unit (231) selects one of them. This selection method is arbitrary. For example, the point with the most nearby points may be selected as the reference point. Once the reference point is set, the point classification unit (231) then sets a point other than that reference point as a prediction point. In this way, a prediction point is selected so that a point exists in the voxel of the upper layer to which the voxel containing a current layer point belongs.

[0206] In step S243, the point classification unit (231) sets a reference point corresponding to each prediction point based on the location information of the resolution of the current layer.

[0207] For example, the point classification unit (231) searches for a prediction point (nearby prediction point) located within a predetermined range from a reference point using location information of the current layer's resolution. The point classification unit (231) corresponds the prediction point to the reference point so that the searched nearby prediction point refers to the reference point. By performing this processing for each reference point, all prediction points are corresponded to the reference point. That is, a reference point corresponding to each prediction point (which each prediction point refers to) is set.

[0208] In step S244, the prediction unit (232) derives a predicted value for each prediction point. That is, the prediction unit (232) derives a predicted value of the attribute information of the prediction point by using the attribute information of the reference point corresponding to each prediction point, based on the correspondence relationship established by the processing of step S243.

[0209] For example, the prediction unit (232) assigns weights to the attribute information of each reference point corresponding to the prediction point, as described with reference to FIG. 4, by weights (α(P,Q(i,j))) based on the inverse of the distance between the prediction point and the reference point, and integrates it by performing an operation as described in Equation (1) above. At that time, the prediction unit (232) derives and applies the distance between the prediction point and the reference point using the position information of the resolution of the current layer.

[0210] In step S245, the operation unit (233) derives the difference value of each prediction point by subtracting the prediction value derived in step S244 from the attribute information of the prediction point selected in step S242.

[0211] In step S246, the update unit (234) derives weights used for update processing. The update unit (234) derives these weights by applying any of the methods listed in the table shown in FIG. 12. That is, the update unit (234) derives weights independently for each layer.

[0212] In step S247, the update unit (234) updates the difference value derived in step S245 using the weight derived in step S246. For example, the update unit (234) performs this update by performing an operation represented by the above-described equation (2).

[0213] In step S248, the operation unit (235) updates the attribute information of the reference point set in step S243 using the difference value updated in step S247.

[0214] In step S249, the hierarchical processing unit (221) determines whether the current layer (current LoD) is the highest layer. Additionally, since the highest layer in the hierarchical structure of attribute information is set independently of the hierarchy of location information, it may differ from the highest layer in the hierarchical structure of location information. If it is determined that the current layer is not the highest layer, the processing proceeds to step S250.

[0215] In step S250, the layer processing unit (221) sets the current layer (current LoD) to be processed to one layer (LoD) above. When the processing of step S250 is finished, the processing returns to step S242. That is, each processing of step S242 to step S250 is executed for each layer.

[0216] And, in step S249, if the current layer is determined to be the highest layer, the layering process is terminated and the process returns to Fig. 18.

[0217] By processing each step in this manner, the hierarchical processing unit (211) can correspond the hierarchical structure of attribute information to the hierarchical structure of location information. In addition, hierarchical processing can be performed using derived weights without relying on lower layers. That is, the hierarchical processing unit (211) can make scalable decoding of attribute information easier.

[0218] <Flow of Quantization Processing>

[0219] Next, an example of the flow of the quantization processing executed in step S222 of FIG. 18 will be explained with reference to the flowchart of FIG. 20. Here, the case in which Method 1-1-2, described as the fourth from the top in the table of FIG. 12, is applied will be explained.

[0220] When quantization processing is initiated, the quantization unit (212) defines a weight derivation function in step S271.

[0221] In step S272, the quantization unit (212) derives weights for each layer (LoD) using the weight derivation function defined in step S271.

[0222] In step S273, the quantization unit (212) quantizes the difference value of the attribute information using the LoD-specific weights derived in step S272.

[0223] In step S274, the quantization unit (212) supplies the weight derivation function definition parameter, which defines the weight derivation function used for weight derivation, to the encoding unit (213) and encodes it. That is, the quantization unit (212) transmits the weight derivation function definition parameter to the decoding side.

[0224] When the processing of step S274 is finished, the quantization processing is finished, and the processing returns to Fig. 18.

[0225] By performing quantization processing in this manner, the quantization unit (212) can perform quantization without relying on the lower layer. Furthermore, inverse quantization without relying on the lower layer becomes possible. That is, the quantization unit (212) can more easily perform scalable decoding of attribute information.

[0226] Flow of Hierarchical Processing

[0227] In addition, even when applying Method 1-1' described as the third from the top in the table of FIG. 6, the stratification processing is performed in a flow that is basically the same as the flowchart of FIG. 19. However, in the processing of step S243, the point classification unit (231) sets a reference point corresponding to each prediction point based on the location information of the resolution of the lowest layer (i.e., the highest resolution). Also, in the processing of step S244, the prediction unit (232) derives the distance between the prediction point and the reference point using the location information of the resolution of the lowest layer (i.e., the highest resolution), and derives a prediction value using a weight based on the inverse of that distance.

[0228] By doing so, the layered processing unit (211) can derive a predicted value of a predicted point using attribute information of a reference point weighted according to distance based on location information of the resolution of the lowest layer of the point cloud.

[0229] Next, an example of the flow of the hierarchical processing when applying Method 1-2, which is described as the fourth from the top of the table in Fig. 6, will be explained with reference to the flowchart in Fig. 21.

[0230] In this case, the hierarchical processing unit (211) performs each of the processing steps S301, S303 to S310 in the same way as each of the processing steps S241, S243 to S250 of FIG. 19.

[0231] Additionally, in step S302, the point classification unit (231) sets all points of the current layer as prediction points, and also sets virtual reference points such that points exist in the voxels of the upper layer to which the voxels containing points of the current layer belong. The point classification unit (231) performs recolor processing using the attribute information of the existing points (prediction points) and generates attribute information of the set virtual reference points.

[0232] In step S303, the point classification unit (231) sets a reference point corresponding to each prediction point based on the location information of the resolution of the current layer among the virtual reference points set in step S302.

[0233] By processing each step in this manner, the hierarchical processing unit (211) can correspond the hierarchical structure of attribute information to the hierarchical structure of location information. In addition, hierarchical processing can be performed using derived weights without relying on lower layers. That is, the hierarchical processing unit (211) can make scalable decoding of attribute information easier.

[0234] <Flow of Quantization Processing>

[0235] Next, an example of the flow of quantization processing when applying Method 1-2-2, which is described as the 7th from the top of the table in FIG. 12, will be explained with reference to the flowchart in FIG. 22.

[0236] When quantization processing is initiated, the quantization unit (212) derives weights for each layer (LoD) in step S321.

[0237] In step S322, the quantization unit (212) quantizes the difference value of the attribute information using the LoD-specific weights derived in step S321.

[0238] In step S323, the quantization unit (212) supplies the weights used for quantization to the encoding unit (213) and encodes them. That is, the quantization unit (212) transmits these weights to the decoding side.

[0239] When the processing of step S323 is finished, the quantization processing is finished, and the processing returns to Fig. 18.

[0240] By performing quantization processing in this manner, the quantization unit (212) can perform quantization without relying on the lower layer. Furthermore, inverse quantization without relying on the lower layer becomes possible. That is, the quantization unit (212) can more easily perform scalable decoding of attribute information.

[0241] <3. Second Embodiment>

[0242] Decoding device

[0243] FIG. 23 is a block diagram illustrating an example of the configuration of a decoding device, which is an embodiment of an information processing device to which the present technology is applied. The decoding device (400) shown in FIG. 23 is a device that decodes encoded data of a point cloud (3D data). The decoding device (400) decodes the encoded data of a point cloud by applying the present technology described above in <1. Scalable Decoding>.

[0244] In addition, FIG. 23 illustrates the main parts, such as the processing unit and the flow of data, and it cannot be said that everything is shown in FIG. 23. That is, in the decoding device (400), there may be a processing unit that is not indicated as a block in FIG. 23, or a processing or flow of data that is not indicated as an arrow in FIG. 23.

[0245] As illustrated in FIG. 23, the decoding device (400) has a decoding target LoD depth setting unit (401), an encoded data extraction unit (402), a location information decoding unit (403), an attribute information decoding unit (404), and a point cloud generation unit (405).

[0246] The decoding target LoD depth setting unit (401) performs processing regarding the setting of the depth of the layer (LoD) to be decoded. For example, the decoding target LoD depth setting unit (401) sets to which layer to decode the encoded data of the point cloud maintained in the encoded data extraction unit (402). The method of setting the depth of the layer to be decoded is arbitrary.

[0247] For example, the decoding target LoD depth setting unit (401) may be set based on instructions regarding layer depth from an external source, such as a user or application. Additionally, the decoding target LoD depth setting unit (401) may determine and set the layer depth to be decoded based on arbitrary information, such as an output image.

[0248] For example, the decoding target LoD depth setting unit (401) may set the layer depth to be decoded based on the viewpoint position, direction, angle of view, and movement of the viewpoint (movement, pan, tilt, zoom), etc. of the 2D image generated from the point cloud.

[0249] Additionally, the data unit for setting the layer depth target for this decoding is arbitrary. For example, the decoding target LoD depth setting unit (401) may set the layer depth for the entire point cloud, set the layer depth for each object, or set the layer depth for each partial area within an object. Of course, the layer depth may also be set for data units other than these examples.

[0250] The encoded data extraction unit (402) acquires and maintains a bit stream input to the decoding device (400). The encoded data extraction unit (402) extracts encoded data of location information and attribute information from the layer specified by the decoding target LoD depth setting unit (401) from the top layer, from the bit stream it maintains. The encoded data extraction unit (402) supplies the extracted encoded data of location information to the location information decoding unit (403). The encoded data extraction unit (402) supplies the extracted encoded data of attribute information to the attribute information decoding unit (404).

[0251] The location information decoding unit (403) acquires encoded data of location information supplied from the encoded data extraction unit (402). The location information decoding unit (403) decodes the encoded data of the location information and generates location information (decoding result). This decoding method is optional as long as it is the same method as that of the location information decoding unit (202) of the encoding device (200). The location information decoding unit (403) supplies the generated location information (decoding result) to the attribute information decoding unit (404) and the point cloud generation unit (405).

[0252] The attribute information decoding unit (404) acquires encoded data of attribute information supplied from the encoded data extraction unit (402). The attribute information decoding unit (404) acquires location information (decoding result) supplied from the location information decoding unit (403). The attribute information decoding unit (404) uses the location information (decoding result) to decode the encoded data of attribute information by the method applying the present technology described above in <1. Scalable Decoding> and generates attribute information (decoding result). The attribute information decoding unit (404) supplies the generated attribute information (decoding result) to the point cloud generation unit (405).

[0253] The point cloud generation unit (405) acquires location information (decoding result) supplied from the location information decoding unit (403). The point cloud generation unit (405) acquires attribute information (decoding result) supplied from the attribute information decoding unit (404). The point cloud generation unit (405) generates a point cloud (decoding result) using the location information (decoding result) and the attribute information (decoding result). The point cloud generation unit (405) outputs the data of the generated point cloud (decoding result) to the outside of the decoding device (400).

[0254] By configuring it in this way, the decoding device (400) can accurately derive a predicted value of attribute information and accurately reverse-layer the hierarchy of attribute information corresponding to the hierarchy of position information. In addition, by doing so, reverse quantization can be performed without relying on the lower layer. That is, the decoding device (400) can more easily perform scalable decoding of attribute information.

[0255] Additionally, these processing units (decoding target LoD depth setting unit (401) to point cloud generation unit (405)) may have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the processing described above. In addition, each processing unit may have, for example, a CPU, ROM, RAM, etc., and realize the processing described above by executing a program using them. Of course, each processing unit may have both configurations, realizing part of the processing described above by a logic circuit and the rest by executing a program. The configurations of each processing unit may be independent of each other, for example, some processing units may realize part of the processing described above by a logic circuit, other processing units may realize the processing described above by executing a program, and yet another processing unit may realize the processing described above by both a logic circuit and the execution of a program.

[0256] <Attribute Information Decoding Section>

[0257] FIG. 24 is a block diagram illustrating a main configuration example of the attribute information decoding unit (404) (Fig. 23). In FIG. 24, the main parts such as the processing unit and data flow are illustrated, and FIG. 24 is not to say that everything is illustrated. That is, in the attribute information decoding unit (404), there may be a processing unit that is not indicated as a block in FIG. 24, or there may be processing or data flow that is not indicated as an arrow, etc. in FIG. 24.

[0258] As illustrated in FIG. 24, the attribute information decoding unit (404) has a decoding unit (421), an inverse quantization unit (422), and an inverse layer processing unit (423).

[0259] The decoding unit (421) acquires encoded data of attribute information supplied to the attribute information decoding unit (404). The decoding unit (421) decodes the encoded data of the attribute information and generates attribute information (decoding result). This decoding method is optional as long as it corresponds to the encoding method of the encoding unit (213) (Fig. 15) of the encoding device (200). Additionally, the generated attribute information (decoding result) corresponds to the attribute information prior to encoding, and as described in the first embodiment, is the difference value between the attribute information and its predicted value, and is quantized. The decoding unit (421) supplies the generated attribute information (decoding result) to the inverse quantization unit (422).

[0260] Additionally, if the encoded data of the attribute information contains information regarding weights (e.g., parameters defining a weight derivation function, weights, etc.), the decoding unit (421) also generates information regarding the weights (decoding result). The decoding unit (421) supplies the generated information regarding the weights (decoding result) to the inverse quantization unit (422). Additionally, the decoding unit (421) may use the generated information regarding the weights (decoding result) for decoding the encoded data of the attribute information.

[0261] The inverse quantization unit (422) acquires attribute information (decoding result) supplied from the decoding unit (421). The inverse quantization unit (422) inversely quantizes the attribute information (decoding result). At that time, the inverse quantization unit (422) applies the technology described above in <1. Scalable Decoding> to perform inverse quantization, which is the reverse processing of quantization by the quantization unit (212) (Fig. 15) of the encoding device (200). That is, the inverse quantization unit (422) inversely quantizes the attribute information (decoding result) using the hierarchical weights as described in <1. Scalable Decoding>.

[0262] Additionally, when information regarding weights (decoding result) is supplied from the decoding unit (421), the inverse quantization unit (422) also acquires the information regarding weights (decoding result). In that case, the inverse quantization unit (422) inversely quantizes the attribute information (decoding result) using the weight specified by the information regarding weights (decoding result), or the weight derived based on the information regarding weights (decoding result).

[0263] The inverse quantization unit (422) supplies the inversely quantized attribute information (decoding result) to the inverse layer processing unit (423).

[0264] The inverse layer processing unit (423) acquires inversely quantized attribute information (decoding result) supplied from the inverse quantization unit (422). As described above, this attribute information is a difference value. Additionally, the inverse layer processing unit (423) acquires position information (decoding result) supplied from the position information decoding unit (403). The inverse layer processing unit (423) uses the position information (decoding result) to perform inverse layering on the acquired attribute information (difference value), which is the reverse processing of layering by the layer processing unit (211) (Fig. 15) of the encoding device (200). At that time, the inverse layer processing unit (423) performs inverse layering by applying the present technology described above in <1. Scalable Decoding>. That is, the inverse layer processing unit (423) derives a prediction value using the position information and restores the attribute information by adding the prediction value to the difference value. In this way, the reverse layering processing unit (423) uses location information to reverse layer attribute information of a hierarchical structure similar to that location information. The reverse layering processing unit (423) supplies the reverse layered attribute information as a decoding result to the point cloud generation unit (405) (Fig. 23).

[0265] By performing inverse quantization or inverse hierarchy as described above, the attribute information decoding unit (404) can accurately derive the predicted value of the attribute information and accurately inversely hierarchy the hierarchy of the attribute information corresponding to the hierarchy of the position information. In addition, inverse quantization can be performed without relying on the lower layer. That is, the decoding device (400) can more easily perform scalable decoding of the attribute information.

[0266] In addition, these processing units (decoding unit (421) to reverse layer processing unit (423)) may have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the processing described above. In addition, each processing unit may have, for example, a CPU, ROM, RAM, etc., and realize the processing described above by executing a program using them. Of course, each processing unit may have both configurations, realizing part of the processing described above by a logic circuit and the rest by executing a program. The configurations of each processing unit may be independent of each other, for example, some processing units may realize part of the processing described above by a logic circuit, other processing units may realize the processing described above by executing a program, and yet another processing unit may realize the processing described above by both a logic circuit and the execution of a program.

[0267] Inverse Hierarchy Processing Unit

[0268] FIG. 25 is a block diagram illustrating a main configuration example of the reverse layered processing unit (423) (Fig. 24). In FIG. 25, the main parts, such as the processing unit and data flow, are illustrated, and it cannot be said that everything illustrated in FIG. 24 is complete. That is, in the reverse layered processing unit (423), there may be a processing unit that is not indicated as a block in FIG. 24, or a processing or data flow that is not indicated as an arrow, etc. in FIG. 24.

[0269] As illustrated in FIG. 24, the reverse layer processing unit (423) has a layer processing unit (441-1), a layer processing unit (441-2), … The layer processing unit (441-1) performs processing on the top layer (0) of the attribute information. The layer processing unit (441-2) performs processing on a layer (1) one level below the top layer of the attribute information. When there is no need to explain these layer processing units (441-1), layer processing units (441-2), … separately, they are referred to as layer processing units (441). That is, the reverse layer processing unit (423) has layer processing units (441) corresponding to the number of layers of attribute information.

[0270] The layer-by-layer processing unit (441) performs processing regarding the inverse layering of attribute information (difference value) of the current layer (current LoD) to be processed. For example, the layer-by-layer processing unit (441) derives a predicted value of attribute information for each point of the current layer and derives attribute information for each point of the current layer by adding the predicted value and the difference value. At that time, the layer-by-layer processing unit (441) performs processing by applying the present technology described above in <1. Scalable Decoding>.

[0271] The hierarchical processing unit (441) has a prediction unit (451), a calculation unit (452), and a merge processing unit (453).

[0272] The prediction unit (451) acquires attribute information L(N) of a reference point of a current layer supplied from the inverse quantization unit (422) or the previous layer processing unit (441). Additionally, the prediction unit (451) acquires location information (decoding result) supplied from the location information decoding unit (403). The prediction unit (451) derives a predicted value P(N) of the attribute information of the prediction point using the location information and attribute information. At that time, the prediction unit (451) derives the predicted value P(N) by applying the present technology described above in <1. Scalable Decoding>. The prediction unit (451) supplies the derived predicted value P(N) to the operation unit (452).

[0273] The operation unit (452) obtains a difference value D(N) of attribute information of a prediction point of a current layer supplied from the inverse quantization unit (422). Additionally, the operation unit (452) obtains a prediction value P(N) of attribute information of a prediction point of a current layer supplied from the prediction unit (451). The operation unit (452) generates (restores) attribute information H(N) of a prediction point by adding the difference value D(N) and the prediction value P(N). The operation unit (452) supplies the generated attribute information H(N) of the prediction point to the merge processing unit (453).

[0274] The merge processing unit (453) acquires attribute information L (N) of a reference point of the current layer supplied from the inverse quantization unit (422) or the preceding layer processing unit (441). Additionally, the merge processing unit (453) acquires attribute information H (N) of a prediction point of the current layer supplied from the operation unit (452). The merge processing unit (453) integrates the acquired attribute information L (N) of the reference point and the attribute information H (N) of the prediction point. By doing so, attribute information of all points from the top layer to the current layer is generated. The merge processing unit (453) supplies the generated attribute information of all points to the subsequent layer processing unit (441) (prediction unit (451) and merge processing unit (453)) as attribute information L (N+1) of a reference point of one lower layer of the current layer.

[0275] In addition, in the case of a layered processing unit (441) in which the lowest layer of the decoding target is the current layer, the merge processing unit (453) supplies attribute information of all generated points to the point cloud generation unit (405) (Fig. 23) as a decoding result.

[0276] By performing reverse layering as described above, the reverse layering processing unit (423) can accurately derive the predicted value of the attribute information and accurately reverse layer the hierarchy of the attribute information corresponding to the hierarchy of the location information. That is, the decoding device (400) can more easily perform scalable decoding of the attribute information.

[0277] Additionally, these processing units (prediction unit (451) to merge processing unit (453)) may have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the processing described above. Also, each processing unit may have, for example, a CPU, ROM, RAM, etc., and realize the processing described above by executing a program using them. Of course, each processing unit may have both configurations, realizing part of the processing described above by a logic circuit and the rest by executing a program. The configurations of each processing unit may be independent of each other, for example, some processing units may realize part of the processing described above by a logic circuit, other processing units may realize the processing described above by executing a program, and yet another processing unit may realize the processing described above by both a logic circuit and the execution of a program.

[0278] <Flow of Decoding Processing>

[0279] Next, the processing performed by this decoding device (400) will be described. The decoding device (400) decodes the encoded data of the point cloud by performing a decoding process. An example of the flow of this decoding process will be explained with reference to the flowchart in FIG. 26.

[0280] When decoding processing is initiated, the decoding target LoD depth setting unit (401) of the decoding device (400) sets the LoD depth to be decoded in step S401.

[0281] In step S402, the encoded data extraction unit (402) acquires and maintains a bit stream and extracts encoded data of position information and attribute information up to the LoD depth set in step S401.

[0282] In step S403, the location information decoding unit (403) decodes the encoded data of the location information extracted in step S402 and generates location information (decoding result).

[0283] In step S404, the attribute information decoding unit (404) decodes the encoded data of the attribute information extracted in step S402 and generates attribute information (decoding result). At that time, the attribute information decoding unit (404) performs processing by applying the present technology described above in <1. Scalable Decoding>. Details of the attribute information decoding processing will be described later.

[0284] In step S405, the point cloud generation unit (405) generates a point cloud (decoding result) using the location information (decoding result) generated in step S403 and the attribute information (decoding result) generated in step S404, and outputs the point cloud (decoding result).

[0285] When the processing of step S405 is finished, the decoding process is terminated.

[0286] By performing the processing of each step in this manner, the decoding device (400) can accurately derive the predicted value of the attribute information and accurately reverse the hierarchy of the attribute information corresponding to the hierarchy of the position information. In addition, by doing so, reverse quantization can be performed without relying on the lower layer. That is, the decoding device (400) can more easily perform scalable decoding of the attribute information.

[0287] <Flow of attribute information decoding processing>

[0288] Next, an example of the flow of attribute information decoding processing executed in step S404 of FIG. 26 will be explained with reference to the flowchart of FIG. 27.

[0289] When the attribute information decoding process is initiated, the decoding unit (421) of the attribute information decoding unit (404) decodes the encoded data of the attribute information in step S421 and generates attribute information (decoding result). This attribute information (decoding result) is quantized as described above.

[0290] In step S422, the inverse quantization unit (422) inversely quantizes the attribute information (decoding result) generated in step S421 by performing an inverse quantization process. At that time, the inverse quantization unit (422) performs inverse quantization by applying the present technology described above in <1. Scalable Decoding>. Details of the inverse quantization process will be described later. Furthermore, this inversely quantized attribute information is the difference between the attribute information and its predicted value, as described above.

[0291] In step S423, the inverse layering processing unit (423) performs inverse layering processing to inverse layer the attribute information (difference value) in step S422 and derives attribute information of each point. At that time, the inverse layering processing unit (423) performs inverse layering by applying the present technology described above in <1. Scalable Decoding>. Details of the inverse layering processing will be described later.

[0292] When the processing of step S423 is finished, the attribute information decoding process is finished, and the process returns to Fig. 26.

[0293] By performing the processing of each step in this manner, the attribute information decoding unit (404) can accurately derive the predicted value of the attribute information and accurately reverse the hierarchy of the attribute information corresponding to the hierarchy of the position information. In addition, by doing so, reverse quantization can be performed without relying on the lower layer. That is, the decoding device (400) can more easily perform scalable decoding of the attribute information.

[0294] <Flow of Inverse Quantization Processing>

[0295] Next, an example of the flow of the inverse quantization process executed in step S422 of FIG. 27 will be explained with reference to the flowchart of FIG. 28. Here, the case in which Method 1-1-2, described as the fourth from the top in the table of FIG. 12, is applied will be explained.

[0296] When the inverse quantization process is initiated, the inverse quantization unit (422) obtains the weight derivation function definition parameters generated by decoding the encoded data in step S441.

[0297] In step S442, the inverse quantization unit (422) defines a weight derivation function based on the function definition parameters obtained in step S441.

[0298] In step S443, the inverse quantization unit (422) derives weights for each layer (LoD) using the weight derivation function defined in step S442.

[0299] In step S444, the inverse quantization unit (422) inversely quantizes the attribute information (decoding result) using the weights derived in step S443.

[0300] When the processing of step S444 is finished, the inverse quantization processing is finished, and the processing returns to Fig. 27.

[0301] By performing inverse quantization processing in this manner, the inverse quantization unit (422) can perform inverse quantization without relying on the lower layer. That is, the decoding device (400) can more easily perform scalable decoding of attribute information.

[0302] <Flow of Reverse Hierarchy Processing>

[0303] Next, an example of the flow of the reverse layering process executed in step S423 of FIG. 27 will be explained with reference to the flowchart in FIG. 29. Here, the case in which Method 1-1, described as the second from the top in the table of FIG. 6, is applied will be explained.

[0304] When the reverse layering process is initiated, the reverse layering processing unit (423) layers the attribute information (decoding result) using the location information (decoding result) in step S461. That is, the reverse layering processing unit (423) performs processing similar to the layering process described in the first embodiment, and corresponds the layered structure of the attribute information with the layered structure of the location information for the layer of the decoding target. By doing so, the attribute information (decoding result) and the location information (decoding result) of each point are corresponded. Based on this correspondence, the following processing is performed.

[0305] In step S462, the layer-by-layer processing unit (441) of the reverse layer processing unit (423) sets the current layer (current LoD) to be processed as the highest layer (LoD).

[0306] In step S463, the prediction unit (451) derives a predicted value for each prediction point. That is, the prediction unit (451) derives a predicted value of the attribute information of the prediction point by using the attribute information of the reference point corresponding to each prediction point.

[0307] For example, the prediction unit (451) derives a predicted value by integrating the attribute information of each reference point corresponding to the prediction point, as described with reference to FIG. 4, by weighting it with a weight (α(P,Q(i,j))) based on the inverse of the distance between the prediction point and the reference point, and by performing an operation as described in Equation (1) above. At that time, the prediction unit (451) derives and applies the distance between the prediction point and the reference point using the location information of the resolution of the current layer (current LoD).

[0308] In step S464, the operation unit (452) restores the attribute information of the prediction point by adding the prediction value derived in step S463 to the difference value of the prediction point.

[0309] In step S465, the merge processing unit (453) merges the attribute information of the derived prediction point and the attribute information of the reference point to generate attribute information of all points of the current layer.

[0310] In step S466, the layer processing unit (441) determines whether the current layer (current LoD) is the lowest layer. If it is determined that the current layer is not the lowest layer, the processing proceeds to step S467.

[0311] In step S467, the layer processing unit (441) sets the current layer (current LoD) to be processed as a layer one level below (LoD). When the processing of step S467 is finished, the processing returns to step S463. That is, each processing of steps S463 through S467 is executed for each layer.

[0312] And, in step S466, if the current layer is determined to be the lowest layer, the reverse layering process is terminated and the process returns to Fig. 27.

[0313] By processing each step in this manner, the reverse layer processing unit (423) can accurately derive the predicted value of the attribute information and accurately reverse layer the hierarchy of the attribute information corresponding to the hierarchy of the location information. That is, the decoding device (400) can more easily perform scalable decoding of the attribute information.

[0314] In addition, when applying Method 1-1' described as the third from the top of the table in Fig. 6, and when applying Method 1-2 described as the fourth from the top of the table in Fig. 6, the inverse layering process is performed in a flowchart basically the same as that of Fig. 29. Therefore, the explanation of these is omitted.

[0315] <Flow of Inverse Quantization Processing>

[0316] Next, an example of the flow of the inverse quantization process when applying Method 1-2-2, which is described as the 7th from the top of the table in FIG. 12, will be explained with reference to the flowchart in FIG. 30.

[0317] When the inverse quantization process is initiated, the inverse quantization unit (422) obtains the layer (LoD) weights generated by decoding the encoded data in step S481.

[0318] In step S482, the inverse quantization unit (422) inversely quantizes the attribute information (decoding result) using the weights obtained in step S481.

[0319] When the processing of step S482 is finished, the inverse quantization processing is finished, and the processing returns to Fig. 27.

[0320] By performing inverse quantization processing in this manner, the inverse quantization unit (422) can perform inverse quantization without relying on the lower layer. That is, the decoding device (400) can more easily perform scalable decoding of attribute information.

[0321] <4. DCM>

[0322] DCM of location information

[0323] As described above, location information is converted into an Octree. In recent years, for example as described in Non-Patent Document 6, a Direct Coding Mode (DCM) has been proposed for nodes in an Octree where the number of leaves (nodes in the lowest layer of the Octree, i.e., points of the highest resolution) is less than a certain number, which encodes the relative distance from that node to each leaf.

[0324] For example, in the Octree conversion of voxel data, if a node to be processed satisfies a predetermined condition and is determined to be sparse, this DCM is applied, and the relative distance (for each direction of xyz) from the node to be processed to each leaf belonging directly or indirectly to that node is calculated and encoded.

[0325] In addition, "directly belonging node" refers to a node that is attached to a relative node in a tree structure. For example, a node directly belonging to a target node represents a node that belongs to the target node and is also one level lower than the target node (so-called child node). In addition, "indirectly belonging node" refers to a node that is attached to a relative node through another node in a tree structure. For example, a node indirectly belonging to a target node represents a node that belongs to the target node through another node and is also two levels or more lower than the target node (e.g., grandchild node).

[0326] For example, as illustrated in Fig. 31, when Octree is formed, if node n0 (LoD=n) is determined to be small, the leaf p0 belonging to (hanging from) node n0 is identified, and the difference (of position) between node n2 and leaf p2 is derived. That is, the encoding of position information of intermediate resolution nodes (nodes of the layer (LoD) between LoD=n and LoD=l) is omitted.

[0327] Similarly, if node n2 (LoD=m) is determined to be small, the leaf p2 belonging to (hanging from) that node n2 is identified, and the difference (of position) between node n2 and leaf p2 is derived. That is, the encoding of position information for intermediate resolution nodes (nodes in the layer (LoD) between LoD=m and LoD=l) is omitted.

[0328] By applying DCM in this way, the creation of small nodes can be omitted, thereby suppressing the increase in processing load for Octreeization.

[0329] <DCM에 대응하는 속성 정보의 계층화>

[0330] In order to enable scalable decoding of attribute information even when such DCM is applied, the attribute information may be hierarchized by forming nodes of intermediate resolution in the Octree of position information as shown in FIG. 32. That is, encoding and decoding of the attribute information using the present technology may be performed without applying DCM.

[0331] In the case of the example of Fig. 32, the attribute information is hierarchized by the existence of intermediate resolution nodes (node ​​n0', node n0" and node n0'") indicated by white circles between node n0 and leaf p0, and intermediate resolution nodes (node ​​n2') indicated by white circles between node n2 and leaf p2. That is, attribute information corresponding to these nodes is generated.

[0332] By doing this, attribute information is encoded even for nodes where location information encoding is omitted by DCM, so attribute information of any level (resolution) can be easily restored without decoding to the highest resolution. In other words, even when DCM is applied, easier and more scalable decoding of attribute information can be realized.

[0333] For example, in the case of an encoding device (200), the attribute information encoding unit (204) (Fig. 14) hierarchizes, quantizes, and encodes the attribute information into a hierarchical structure as shown in the example of Fig. 32 for position information of a hierarchical structure as shown in the example of Fig. 31. That is, the hierarchical processing unit (211) (Fig. 15) generates attribute information of a hierarchical structure as shown in the example of Fig. 32.

[0334] In addition, for example, in the case of a decoding device (400), the attribute information decoding unit (404) (Fig. 23) decodes the encoded data of the attribute information to a desired resolution (up to a desired layer). As described above, the attribute information in the encoded data has a layered structure such as the example in Fig. 32. Accordingly, the attribute information decoding unit (404) (reverse layering processing unit (423) (Fig. 24)) can restore the attribute information of any layer by reversing the layer without requiring decoding up to the highest resolution.

[0335] <Node Resolution Setting 1>

[0336] In addition, when referencing location information in processing such attribute information, intermediate resolution nodes for which the encoding of location information is omitted by DCM may be represented, for example, at the resolution of that layer.

[0337] <In the case of encoding>

[0338] For example, the case in which the LoD=m layer is processed for encoding point cloud data is described. The location information encoding unit (201) applies DCM to the nodes and generates location information with a layered structure as shown in the example of FIG. 31, and then encodes it. That is, the location information of the node n0" of LoD=m corresponding to node n0 is not included in the encoded data of this location information.

[0339] In contrast, the attribute information encoding unit (204) generates attribute information of node n0" of LoD=m corresponding to node n0, which is omitted by DCM as in the example shown in FIG. 33. At that time, the attribute information encoding unit (204) (layered processing unit (211)) represents the node n0" at the resolution of that layer (LoD=m). That is, the attribute information encoding unit (204) (layered processing unit (211)) generates attribute information corresponding to node n0" at the resolution of that layer (LoD=m).

[0340] In that case, the method for generating attribute information corresponding to node n0" at the resolution of the hierarchy (LoD=m) is arbitrary. For example, attribute information corresponding to node n0 or leaf p0 may be used to generate attribute information corresponding to node n0". In that case, for example, attribute information of node n0 or leaf p0 may be transformed according to the difference in resolution between node n0 or leaf p0 and node n0" to generate attribute information corresponding to node n0". Additionally, for example, attribute information of neighboring nodes may be used to generate attribute information corresponding to node n0".

[0341] By doing so, the encoding device (200) can encode attribute information corresponding to the node n0" of the resolution of the layer (LoD=m).

[0342] <In the case of decoding>

[0343] Next, the case in which the layer of LoD=m is the processing target for decoding the encoded data of the point cloud data will be explained. As described above, the encoded data of the encoded attribute information contains attribute information corresponding to the node n0" of the layer (LoD=m) resolution. Therefore, the attribute information decoding unit (404) can easily restore the attribute information corresponding to the node n0" of the layer (LoD=m) resolution by decoding the encoded data of the attribute information up to the layer (LoD=m).

[0344] That is, the decoding device (400) can more easily decode attribute information scalably, even when DCM is applied.

[0345] In addition, regarding location information, location information corresponding to node n0" of the layer (LoD=m) is not encoded. Therefore, the location information decoding unit (403) may generate location information corresponding to node n0" of the layer (LoD=m) resolution by restoring location information corresponding to leaf p0 as in the example shown in FIG. 34 and converting the resolution from the final resolution (highest resolution) to the resolution of the layer (LoD=m).

[0346] <Node Resolution Settings 2>

[0347] Additionally, intermediate resolution nodes for which position information encoding is omitted by DCM may be represented, for example, at the highest resolution (i.e., the resolution of the leaf (LoD=l)).

[0348] <In the case of encoding>

[0349] For example, the case in which the layer of LoD=m is processed for encoding point cloud data is described. In this case, the attribute information encoding unit (204) generates attribute information of node n0" of LoD=m corresponding to node n0, which is omitted by DCM as shown in the example in FIG. 35. At that time, the attribute information encoding unit (204) (layering processing unit (211)) represents that node n0" at the resolution of the leaf (LoD=l). That is, the attribute information encoding unit (204) (layering processing unit (211)) generates attribute information corresponding to node n0" at the resolution of the leaf (LoD=l) (i.e., attribute information identical to the attribute information corresponding to the leaf p0).

[0350] By doing this, the encoding device (200) can encode attribute information corresponding to the node n0" of the resolution of the leaf (LoD=l).

[0351] <In the case of decoding>

[0352] Next, the case in which the layer of LoD=m is the processing target for decoding the encoded data of the point cloud data will be explained. As described above, the encoded data of the encoded attribute information contains attribute information corresponding to the node n0" of the leaf (LoD=l) resolution. Therefore, the attribute information decoding unit (404) can easily recover the attribute information corresponding to the node n0" of the leaf (LoD=l) resolution by decoding the encoded data of the attribute information up to the layer (LoD=m).

[0353] That is, the decoding device (400) can more easily decode attribute information scalably, even when DCM is applied.

[0354] In addition, regarding location information, the node n0" of the layer (LoD=m) is not encoded. Therefore, the location information decoding unit (403) may restore location information corresponding to the leaf p0 as in the example shown in FIG. 36, and use it as location information corresponding to the node n0" of the leaf (LoD=l) resolution.

[0355] <5. QuantizationWeight>

[0356] <Weight Derivation Function Example 1>

[0357] In <1. Scalable Decoding>, examples of quantization weights were described with reference to FIG. 12 and others. For example, in <Weights: Method 1-1-1>, an example of deriving weights using a predetermined weight derivation function was described. The weight derivation function for deriving these weights may be any function, but may be, for example, a function such as Equation (11) and Equation (12) as shown in A of FIG. 37. Equation (11) and Equation (12) are listed again below.

[0358]

[0359]

[0360] In Equations (11) and (12), LoDCount represents the lowest layer (highest resolution layer) of the attribute information hierarchy. Additionally, QuantizationWeight[i] represents the weight (this weight is a value set per layer) in layer i of this hierarchy (the (i-1)th layer from the highest layer (lowest resolution layer)). pointCount represents the total number of nodes in this hierarchy, i.e., the total number of points. predictorCount[i] represents the number of predicted points in layer i, i.e., the number of points from which a predicted value is derived in layer i. The cumulative value of the number of predicted points in each layer from layer 0 (highest layer) to layer LoDCount (lowest layer) (the numerator of the rightmost side of Equation (11)) becomes the total number of points (pointCount).

[0361] As shown in Equation (11), the weight (QuantizationWeight[i]) of layer i in this case is set to the ratio of the total number of points to the number of predicted points of layer i. However, as shown in Equation (12), for the lowest layer (i==LoDCount), the weight (QuantizationWeight[i]) is set to "1".

[0362] By deriving the weight (QuantizationWeight[i]) using such a weight derivation function, the encoding device (200) and the decoding device (400) can derive the weight without relying on the lower layer. Therefore, as described above in <Weight: Method 1>, quantization and inverse quantization without relying on the lower layer become possible. That is, as described above, since reference to information in the lower layer is unnecessary, the encoding device (200) and the decoding device (400) can more easily realize scalable decoding.

[0363] In addition, by using this weight derivation function (Equation (11) and Equation (12)), the encoding device (200) and the decoding device (400) can derive weights such that the value increases as the layer gets higher. Therefore, the reduction in the precision of encoding and decoding can be suppressed.

[0364] In addition, as described above in <Weight: Method 1-1-1>, the encoding device (200) and the decoding device (400) may share this weight derivation function (Equation (11) and Equation (12)) in advance. By doing so, the encoding device (200) and the decoding device (400) can derive the same weight. Furthermore, since there is no need to transmit information to share this weight derivation function, the reduction in encoding efficiency and the increase in the load of encoding processing and decoding processing can be suppressed.

[0365] In addition, as described above in <Weight: Method 1-1-2>, the encoding device (200) may define the weight derivation function (Equation (11) and Equation (12)) during encoding. In that case, the encoding device (200) may transmit the defined weight derivation function, or the parameter (weight derivation function definition parameter) that enables the decoding device (400) to derive the weight derivation function, to the decoding device (400).

[0366] For example, the encoding device (200) may include such weight derivation function or weight derivation function definition parameter in a bit stream containing the encoded data of the point cloud and transmit it to the decoding device (400).

[0367] For example, in the quantization process described with reference to the flowchart of FIG. 20, the quantization unit (212) defines the above-described equations (11) and (12) as weight derivation functions (step S271). Then, the quantization unit (212) derives weights for each layer (LoD) using the weight derivation function (step S272). Then, the quantization unit (212) quantizes the difference value of the attribute information using the derived weights for each LoD (step S273). Then, the quantization unit (212) supplies the weight derivation function or the weight derivation function definition parameter to the encoding unit (213) and encodes it (step S274). That is, the quantization unit (212) transmits the weight derivation function or the weight derivation function definition parameter to the decoding side.

[0368] Additionally, the encoding device (200) may transmit this weight derivation function or weight derivation function definition parameter to the decoding device (400) as data or a file independent of the bit stream containing the encoded data of the point cloud.

[0369] In this way, by transmitting such weight derivation functions or parameters from the encoding device (200) to the decoding device (400), the encoding device (200) and the decoding device (400) can share the weight derivation functions (Equation (11) and Equation (12)). Accordingly, the encoding device (200) and the decoding device (400) can derive the same weights from each other.

[0370] In addition, as described above in <Weight: Method 1-2>, the encoding device (200) and the decoding device (400) may share the weight (QuantizationWeight[i]) derived using this weight derivation function (Equation (11) and Equation (12)). For example, as described above in <Weight: Method 1-2-2>, the encoding device (200) may derive the weight using this weight derivation function during encoding and transmit the weight to the decoding device (400).

[0371] For example, the encoding device (200) may include these weights in a bit stream containing encoded data of a point cloud and transmit it to the decoding device (400).

[0372] For example, in the quantization process described with reference to the flowchart of FIG. 22, the quantization unit (212) derives a weight for each layer (LoD) using the weight derivation function (Equation (11) and Equation (12)) described above (Step S321). Then, the quantization unit (212) quantizes the difference value of the attribute information using the derived weight for each LoD (Step S322). Then, the quantization unit (212) supplies the weight used for the quantization to the encoding unit (213) and encodes it (Step S323). That is, the quantization unit (212) transmits this weight to the decoding side.

[0373] Additionally, the encoding device (200) may transmit this weight to the decoding device (400) as data or a file independent of the bit stream containing the encoded data of the point cloud.

[0374] In this way, by transmitting weights from the encoding device (200) to the decoding device (400), the encoding device (200) and the decoding device (400) can share weights. Therefore, the encoding device (200) and the decoding device (400) can easily derive weights without relying on lower layers. Thus, the encoding device (200) and the decoding device (400) can more easily realize scalable decoding. In addition, since the decoding device (400) does not need to derive weights using a weight derivation function, it can suppress an increase in the load of decoding processing.

[0375] <Weight Derivation Function Example 2>

[0376] The weight derivation function may be a function such as Equation (13), as shown in B of FIG. 37, for example. Equation (13) is listed again below.

[0377]

[0378] As with Equations (11) and (12), in Equation (13), LoDCount represents the lowest layer of the hierarchy of attribute information. Additionally, QuantizationWeight[i] represents the weight in layer i of this hierarchy. PredictorCount[i] represents the number of prediction points in layer i, that is, the number of points from which a prediction value is derived in layer i. The accumulated value of the number of prediction points in each layer from layer i (the layer to be processed) to layer LoDCount (the lowest layer) (the numerator of the rightmost side of Equation (11)) becomes the number of points in layer i or lower.

[0379] As shown in Equation (13), in this case, the weight of layer i (QuantizationWeight[i]) is set as the ratio of the number of points below layer i to the number of predicted points of layer i.

[0380] Even when applying the weight derivation function of this equation (13), just as when applying the weight derivation functions of equations (11) and (12), the encoding device (200) and the decoding device (400) can derive weights without relying on the lower layer. Therefore, as described above in <Weight: Method 1>, quantization and inverse quantization without relying on the lower layer become possible. That is, as described above, since reference to information of the lower layer is unnecessary, the encoding device (200) and the decoding device (400) can more easily realize scalable decoding.

[0381] In addition, when applying the weight derivation function of this equation (13), just as when applying the weight derivation functions of equations (11) and (12), the encoding device (200) and the decoding device (400) can derive weights such that the value increases as the layer gets higher. Therefore, the reduction in the precision of encoding and decoding can be suppressed.

[0382] As illustrated in FIG. 5, the weight (QuantizationWeight[i]) is set according to the value that points are more important (having a greater influence on subjective image quality) as they have a higher chance of being referenced by other points. In general, there is a tendency for prediction points in higher layers to have a higher chance of being referenced by other points. Therefore, higher prediction points are given larger weights. As described above with reference to FIG. 12, even when setting weights by layer (LoD), higher layers are given larger weights according to this value.

[0383] In the case of a point cloud where points are sufficiently dense—that is, where other points exist in the vicinity of each point—the number of points generally increases as the resolution increases. Typically, the degree of increase in the number of points is sufficiently large, and the number of predicted points (predictorCount[i]) increases monotonically toward the lower layers. Therefore, whether the weight derivation function of Equation (11) and Equation (12) or the weight derivation function of Equation (13) is applied, the weight of layer i (QuantizationWeight[i]) tends to increase as it moves toward the upper layers.

[0384] In contrast, for point clouds with a low point count—that is, when the probability of other points existing near each point is low—increasing the resolution generally does not significantly increase the number of points. Typically, a large proportion of points are assigned to prediction points in the upper layers of the hierarchy, potentially leading to a decrease in the number of prediction points as you move toward the lower layers.

[0385] When applying the weight derivation function of Equations (11) and (12), the weight of layer i (QuantizationWeight[i]) becomes larger as the number of prediction points (predictorCount[i]) in that layer decreases. Therefore, typically, it is conceivable that the upper layer, where a large proportion of points are assigned to prediction points, may end up having a smaller weight than the lower layer. In other words, the correspondence between the weight and the number of opportunities for a prediction point to be referenced weakens, and there is a possibility that subjective image quality degradation due to quantization may increase.

[0386] In contrast, the number of points below layer i increases monotonically toward the upper layers. Therefore, when applying the weight derivation function of Equation (13), the weight of layer i (QuantizationWeight[i]) is likely to increase as it moves toward the upper layers. In other words, it is easy to maintain the tendency to set a large weight for prediction points (of higher layers) that have a greater chance of being referenced by other points. Consequently, subjective image quality degradation due to quantization can be suppressed compared to when applying the weight derivation functions of Equations (11) and (12).

[0387] In addition, as described above in <Weight: Method 1-1-1>, the encoding device (200) and the decoding device (400) may share this weight derivation function (Equation (13)) in advance. By doing so, the encoding device (200) and the decoding device (400) can derive the same weight. Furthermore, since there is no need to transmit information to share this weight derivation function, the reduction in encoding efficiency and the increase in the load of encoding processing and decoding processing can be suppressed.

[0388] In addition, as described above in <Weight: Method 1-1-2>, the encoding device (200) may define this weight derivation function (Equation (13)) during encoding. In that case, the encoding device (200) may transmit the defined weight derivation function, or the parameter (weight derivation function definition parameter) that enables the decoding device (400) to derive the weight derivation function, to the decoding device (400).

[0389] For example, the encoding device (200) may include such weight derivation function or weight derivation function definition parameter in a bit stream containing the encoded data of the point cloud and transmit it to the decoding device (400).

[0390] For example, in the quantization process described with reference to the flowchart of FIG. 20, the quantization unit (212) defines the above-described equation (13) as a weight derivation function (step S271). Then, the quantization unit (212) derives a weight for each layer (LoD) using the weight derivation function (step S272). Then, the quantization unit (212) quantizes the difference value of the attribute information using the derived weight for each LoD (step S273). Then, the quantization unit (212) supplies the weight derivation function or the weight derivation function definition parameter to the encoding unit (213) and encodes it (step S274). That is, the quantization unit (212) transmits the weight derivation function or the weight derivation function definition parameter to the decoding side.

[0391] Additionally, the encoding device (200) may transmit this weight derivation function or weight derivation function definition parameter to the decoding device (400) as data or a file independent of the bit stream containing the encoded data of the point cloud.

[0392] In this way, by transmitting such a weight derivation function or parameter from the encoding device (200) to the decoder (400), the encoding device (200) and the decoder (400) can share the weight derivation function (Equation (13)). Accordingly, the encoding device (200) and the decoder (400) can derive the same weight.

[0393] In addition, as described above in <Weight: Method 1-2>, the encoding device (200) and the decoding device (400) may share the weight (QuantizationWeight[i]) derived using this weight derivation function (Equation (13)). For example, as described above in <Weight: Method 1-2-2>, the encoding device (200) may derive the weight using this weight derivation function during encoding and transmit the weight to the decoding device (400).

[0394] For example, the encoding device (200) may include these weights in a bit stream containing encoded data of a point cloud and transmit it to the decoding device (400).

[0395] For example, in the quantization process described with reference to the flowchart of FIG. 22, the quantization unit (212) derives a weight for each layer (LoD) using the weight derivation function (Equation (13)) described above (Step S321). Then, the quantization unit (212) quantizes the difference value of the attribute information using the derived weight for each LoD (Step S322). Then, the quantization unit (212) supplies the weight used for the quantization to the encoding unit (213) and encodes it (Step S323). That is, the quantization unit (212) transmits this weight to the decoding side.

[0396] Additionally, the encoding device (200) may transmit this weight to the decoding device (400) as data or a file independent of the bit stream containing the encoded data of the point cloud.

[0397] In this way, by transmitting these weights from the encoding device (200) to the decoding device (400), the encoding device (200) and the decoding device (400) can share the weights. Accordingly, the encoding device (200) and the decoding device (400) can easily derive weights without relying on lower layers. Therefore, the encoding device (200) and the decoding device (400) can more easily realize scalable decoding. In addition, since the decoding device (400) does not need to derive weights using a weight derivation function, it can suppress an increase in the load of decoding processing.

[0398] <6. LoDGeneration>

[0399] LoDGeneration during location information decryption

[0400] As described above with reference to Fig. 6 in <1. Scalable Decoding>, scalable decoding of attribute information is made possible by hierarchizing attribute information to have a structure similar to the hierarchical structure of position information. Furthermore, the hierarchical structure of attribute information is established by selecting prediction points (or reference points) of each layer. The process of establishing such a hierarchical structure is called LoD Generation processing or SubSampling processing.

[0401] As described above in <2. First Embodiment> or <3. Second Embodiment>, this LoD generation processing is performed during the encoding and decoding of attribute information. For example, in the case of the encoding device (200), the point classification unit (231) of the attribute information encoding unit (204) performs this LoD generation processing (step S242). Also, in the case of the decoding device (400), the reverse layer processing unit (423) of the attribute information decoding unit (404) performs this LoD generation processing (step S461).

[0402] This LoD generation process may be performed, for example, during the decoding of encoded data of location information, in addition to this example. By decoding the encoded data of location information, the hierarchical structure of the location information (which layer each point belongs to) is revealed. Therefore, as a LoD generation process, information representing this hierarchical structure of location information is generated. Then, based on that information, the hierarchy and inverse hierarchy of attribute information are performed to correspond to the hierarchical structure of the location information. By doing so, the hierarchical structure of the location information can be applied to the attribute information.

[0403] In addition, encoding and decoding of attribute information are performed by applying such hierarchical and inverse hierarchical structures. That is, encoding and decoding are performed in a state where the hierarchical structure of position information and attribute information corresponds. By doing so, scalable decoding of attribute information becomes possible. Furthermore, by reducing excessive processing, the increase in load for encoding and decoding of attribute information can be suppressed.

[0404] Additionally, in the case of the encoding device (200), this LoD generation processing may be performed during the encoding of position information.

[0405] <LoDGeneration 처리의 예>

[0406] This LoD generation process is described. When the encoded data of location information is decoded, a hierarchical structure, such as an Octree, is restored. FIG. 38 illustrates an example of the hierarchical structure of the restored location information. The nodes (611) indicated by circles in this hierarchical structure represent the hierarchical points. Additionally, FIG. 38 illustrates only a part of the hierarchical structure (a part of the nodes (611)). Although nodes (611-1) to (611-12) are illustrated in FIG. 38, they are referred to as nodes (611) when there is no need to distinguish and explain each node separately.

[0407] Location information is decoded layer by layer from the highest layer (LoDIdx=0) toward the lowest layer. In the example shown in FIG. 38, a node (611-1) appears in the highest layer (LoDIdx=0). In the one lower layer (LoDIdx=1), as the resolution increases, the point corresponding to the node (611-1) is divided, and a new node (611-2) appears. In the one lower layer (LoDIdx=2), the point corresponding to the node (611-1) is divided, and a new node (611-3) and a node (611-4) appear, and the point corresponding to the node (611-2) is divided, and a new node (611-5) to a node (611-7) appear.

[0408] In that one lower layer (LoDIdx=3), the point corresponding to node (611-1) is divided, and a new node (611-7) appears. In that one lower layer (LoDIdx=4), the point corresponding to node (611-1) is divided, and new nodes (611-8) to (611-10) appear, and the point corresponding to node (611-7) is divided, and new nodes (611-11) and (611-12) appear.

[0409] For each node (611) that appears sequentially in this manner, a label indicating a hierarchy (LodIdx) is attached using, for example, a list of data structures. An example of label attachment for the example of FIG. 38 is illustrated in FIG. 39. This label attachment is performed hierarchically in order from the top layer of the hierarchical structure of location information toward the bottom layer, and for each newly appearing node, a label with a value indicating that layer is attached.

[0410] For example, when the top layer (LoDIdx=0) is processed and a node (611-1) appears, as shown in the first layer from the top (layer of “1”) of FIG. 39, a label of “0” is attached to that node (611-1) in the list.

[0411] When one lower layer (LoDIdx=1) is processed and a new node (611-2) appears, as shown in the second row from the top (row of “2”) of FIG. 39, the label “1” is attached to that node (611-2) in the list. Since the node (611-1) already has a label attached, no label is attached here.

[0412] Additionally, when one lower layer (LoDIdx=2) is processed and nodes (611-3) to (611-6) appear anew, as shown in the third row from the top (row "3") of FIG. 39, a label of "2" is attached to each of the nodes (611-3) to (611-6) in the list. Since nodes (611-1) and (611-2) already have labels attached, no labels are attached here. Also, in the hierarchical structure (Octree), each node (each point) is arranged in a morton order. Likewise, in the list, the labels of each node (611) are sorted in a morton order. Therefore, in the example of FIG. 39, the arrangement of labels becomes "0, 2, 2, 1, 2, 2, …".

[0413] Additionally, when one lower layer (LoDIdx=3) is processed and a new node (611-7) appears, the label "3" is attached to that node (611-7) in the list, as shown in the fourth row from the top (row "4") of FIG. 39. Since node (611-1) already has a label attached, no label is attached here. Since node (611-7) appears after node (611-1) in the order of the second, in the case of the example of FIG. 39, the arrangement of the labels becomes "0, 3, …".

[0414] Additionally, when one lower layer (LoDIdx=4) is processed and nodes (611-8) to (611-12) newly appear, as shown in the 5th row from the top (row "5") of FIG. 39, the label "4" is attached to these nodes (611-8) to (611-12) in the list. Since nodes (611-1) and (611-7) already have labels attached, no labels are attached here. In the order of first, nodes (611-8) to (611-10) appear after node (611-1), and nodes (611-11) and (611-12) appear after node (611-7). Therefore, in the case of the example of FIG. 39, the arrangement of labels is "0, 4, 4, 4, 3, 4, 4, …" It becomes like 」.

[0415] By referencing the list generated as described above to hierarchize attribute information and performing encoding and decoding of the attribute information, attribute information can be encoded and decoded in a hierarchical structure similar to that of position information. Therefore, scalable decoding of attribute information can be realized. Furthermore, by reducing excessive processing, the increase in load for encoding and decoding of attribute information can be suppressed.

[0416] <DCM을 포함하는 경우 그 1>

[0417] A case in which DCM is included in the hierarchical structure of location information is explained. An example of the hierarchical structure of location information in that case is illustrated in FIG. 40. In the hierarchical structure illustrated in FIG. 40, the point of node (611-21) appearing in the layer of LoDIdx=1 is divided in the layer of LoDIdx=3. That is, a new node (611-22) appears in the layer of LoDIdx=3. In addition, DCM is applied to this node (611-21). DCM is also applied to node (611-22).

[0418] For nodes to which such DCM applies, labeling may be performed using a different list than for nodes to which DCM does not apply. An example of labeling for the example of FIG. 40 by this method is illustrated in FIG. 41. As shown in FIG. 41, for nodes to which DCM does not apply, labeling is performed using List A, as in the case of the example of FIG. 39. In contrast, for nodes (611-21) and (611-22) to which DCM applies, labeling is performed using List B, which is different from List A. Additionally, List A and List B are names assigned to each list for convenience of explanation, and the actual names are arbitrary.

[0419] After the top layer (LoDIdx=0) is processed, the one lower layer (LoDIdx=1) is processed, and when a new node (611-21) with DCM applied appears, the label “1” is attached to that node (611-21) in List B, which is shown on the right side of the drawing from the dotted line of FIG. 41. Since the node (611-22) also has DCM applied and is encoded as a relative distance from this node (611-21) with LoDIdx=1, the label is attached to the node (611-22) in this layer. However, since the node (611-22) is separated from the node (611-21) with LoDIdx=3, the label “3” is attached to this node (611-22).

[0420] Additionally, the labels for nodes (611-21) and nodes (611-22) are arranged according to the positions of both nodes in the Morton order of the lowest layer (highest resolution). In the case of the example of FIG. 40, since the nodes (611-21) and nodes (611-22) are arranged in order, the labels are arranged in the order of “1, 3”.

[0421] Labels are similarly set in List B for nodes (611) to which other DCMs are applied. Then, in encoding and decoding attribute information, the label columns of List A and List B are used. At that time, the label columns of List A and List B are merged and sorted in Morton order. That is, the labels are sorted to correspond to the nodes arranged in Morton order at the lowest level (highest resolution).

[0422] Additionally, for the labels of List B (labels of nodes to which DCM is applied), a process to remove duplicate points is performed. For the nodes (611) to which DCM is applied, labels are attached to the nodes (611) that are separated at the time of their first appearance. For example, in the case of FIG. 41, labels are attached to both nodes (611-21) and nodes (611-22) in the hierarchy where LoDIdx=1.

[0423] However, depending on the layer of the decoding target, there may be cases where the nodes are not separated. For example, when decoding at the layer (resolution) of LoDIdx=2 in the example of FIG. 40, nodes (611-21) and (611-22) are not separated. That is, in the layer of LoDIdx=2, the point (node ​​(611-21)) corresponding to the label "1" and the point (node ​​(611-22)) corresponding to the label "3" overlap. Therefore, the label for node (611-22) is unnecessary. Thus, as described above, the label with the overlapping corresponding point is removed.

[0424] That is, this process of removing duplicate points is performed only on the labels of List B (labels of nodes to which DCM is applied) (labels of List A are excluded).

[0425] As described above, a label column corresponding to a node where DCM is not applied and a label column corresponding to a node where DCM is applied are merged, each label is sorted in Morton order, and a single label column with duplicate points appropriately removed is applied to the encoding and decoding of attribute information as a result of LoD generation processing.

[0426] By referencing the list generated as described above to hierarchize attribute information and performing encoding and decoding of the attribute information, attribute information can be encoded and decoded in a hierarchical structure similar to that of location information. Therefore, scalable decoding of attribute information can be realized even when DCM is included in the hierarchical structure of location information. Furthermore, by reducing excessive processing, the increase in load for encoding and decoding of attribute information can be suppressed.

[0427] <DCM을 포함하는 경우 그 2>

[0428] In addition, even when DCM is included in the hierarchical structure of location information, the nodes to which DCM is applied and the nodes to which DCM is not applied are not separated; that is, labeling may be performed using the same list. FIG. 42 illustrates an example of labeling for the hierarchical structure shown in FIG. 40, similar to FIG. 41. In the example of FIG. 42, labeling is performed using a single list, similar to the example of FIG. 39.

[0429] First, the top layer (LoDIdx=0) is processed, and when a node (611-1) appears, a label of “0” is attached to that node (611-1) in the list, just as in the example of FIG. 39 (the first row from the top (row of “1”) in FIG. 42).

[0430] When that one lower layer (LoDIdx=1) is processed and a new node (611-21) with DCM applied appears, the label "1" is attached to that node (611-21) in the list. Node (611-22) also has DCM applied and is encoded as a relative distance from this node (611-21) with LoDIdx=1, so the label is attached to the node (611-22) in this layer. However, since node (611-22) is separated from node (611-21) at LoDIdx=3, the label "3" is attached to this node (611-22) (the second layer from the top in FIG. 42 (the layer of "2")).

[0431] In addition, as in the case of FIG. 41, the labels for nodes (611-21) and nodes (611-22) are arranged according to the positions of both nodes in the Morton order of the lowest layer (highest resolution). In the case of the example of FIG. 40, since the nodes (611-21) and nodes (611-22) are arranged in order, the labels are arranged in the order of “1, 3”.

[0432] However, in the case of the example of FIG. 42, the removal of duplicate points is performed whenever labeling is applied to the node to which DCM is applied. For example, when decoding at the layer (resolution) of LoDIdx=2 in the example of FIG. 40, the point (node ​​(611-21)) corresponding to the label "1" and the point (node ​​(611-22)) corresponding to the label "3" overlap, so the label "3" is removed. In other words, in the second layer from the top (layer of "2") of FIG. 42, only the labeling of "1" is performed.

[0433] In contrast, when decoding at the layer (resolution) of LoDIdx=4 in the example of FIG. 40, the point (node ​​(611-21)) corresponding to the label "1" and the point (node ​​(611-22)) corresponding to the label "3" do not overlap, so the removal of the label is not performed. In other words, in the second layer from the top (layer of "2") of FIG. 42, the attachment of the label "1" and the attachment of the label "3" are performed.

[0434] Additionally, when one lower layer (LoDIdx=2) is processed and a new node (611) appears, the label “2” is attached to that node in the list (the third layer from the top in FIG. 42 (the “3” layer)). Likewise, when one more lower layer (LoDIdx=3) is processed and a new node (611) appears, the label “3” is attached to that node in the list (the third layer from the top in FIG. 42 (the “4” layer)). Likewise, when one more lower layer (LoDIdx=4) is processed and a new node (611) appears, the label “4” is attached to that node in the list (the third layer from the top in FIG. 42 (the “5” layer)). These labels are sorted in a morton order, just like the nodes (611).

[0435] A single label column, generated as described above and containing a label corresponding to a node to which DCM is not applied and a label corresponding to a node to which DCM is applied, is applied to the encoding and decoding of attribute information as a result of LoD generation processing. In this method, the generated label column has each label sorted in a morton order. Therefore, this label column can be applied to the encoding and decoding of attribute information without the need to sort each label in a morton order and merge them, as in the example of FIG. 41.

[0436] By referencing the list generated as described above to hierarchize attribute information and performing encoding and decoding of the attribute information, attribute information can be encoded and decoded in a hierarchical structure similar to that of position information. Therefore, this method also enables scalable decoding of attribute information. Furthermore, by reducing excessive processing, the increase in load for encoding and decoding of attribute information can be suppressed.

[0437] Encoding device

[0438] FIG. 43 is a block diagram illustrating an example of the configuration of the encoding device (200) in this case. That is, the encoding device (200) shown in FIG. 43 encodes a point cloud by applying the technology described above in <6. LoDGeneration>.

[0439] In addition, FIG. 43 illustrates the main parts, such as processing units and data flow, and it cannot be said that everything is shown in FIG. 43. That is, in the encoding device (200), there may be processing units that are not indicated as blocks in FIG. 43, or processing or data flow that is not indicated as arrows in FIG. 43.

[0440] The encoding device (200) illustrated in FIG. 43 has a position information encoding unit (201) and a bit stream generating unit (205), as in FIG. 14. However, when the position information decoding unit (202) decodes the encoded data of the position information, it performs LoD generation processing (subsampling processing) using a list as described above with reference to FIG. 38 to FIG. 42.

[0441] For example, if the hierarchical structure of location information does not include a node to which DCM is applied, the location information decoding unit (202) performs LoD generation processing by the method described with reference to FIG. 39. In addition, for example, if the hierarchical structure of location information includes a node to which DCM is applied, the location information decoding unit (202) performs LoD generation processing by the method described with reference to FIG. 41 or FIG. 42.

[0442] Then, the location information decoding unit (202) supplies the generated list (label column) to the attribute information encoding unit (204). The attribute information encoding unit (204) encodes the attribute information using the LoD generation processing result (label column). That is, the attribute information encoding unit (204) performs encoding by applying the label column (hierarchical structure of location information) as the hierarchical structure of the attribute information. By doing so, the attribute information encoding unit (204) can hierarchize each attribute information into a hierarchical structure similar to the location information without performing LoD generation processing (subsampling processing).

[0443] Accordingly, the encoding device (200) can realize scalable decoding of attribute information. In addition, the encoding device (200) can reduce excessive processing and suppress an increase in the load of encoding processing of attribute information.

[0444] <Flow of Encoding Processing>

[0445] An example of the flow of the encoding process in this case is explained with reference to the flowchart in FIG. 44. When the encoding process is initiated, the position information encoding unit (201) of the encoding device (200) encodes the position information of the input point cloud in step S601 and generates encoded data of the position information.

[0446] In step S602, the location information decoding unit (202) decodes the encoded data of the location information generated in step S601 and generates location information. At that time, the location information decoding unit (202) performs LoD generation processing using a list as described above with reference to FIGS. 38 to 42 and performs label attachment for each node.

[0447] In step S603, the point cloud generation unit (203) uses the attribute information of the input point cloud and the location information (decoding result) generated in step S202 to perform recolor processing and correspond the attribute information to the location information.

[0448] In step S604, the attribute information encoding unit (204) encodes the attribute information recolored in step S603 by executing attribute information encoding processing and generates encoded data of the attribute information. At that time, the attribute information encoding unit (604) encodes the attribute information using the result (label column) of the LoD generation processing performed in step S602. In this case as well, the attribute information encoding processing is performed in the same flow as the flowchart of FIG. 18. However, details of the layering processing executed in step S221 will be described later.

[0449] In step S605, the bit stream generating unit (205) generates and outputs a bit stream including encoded data of position information generated in step S601 and encoded data of attribute information generated in step S604.

[0450] When the processing of step S605 is finished, the encoding process is terminated.

[0451] By performing the processing of each step in this manner, the encoding device (200) can correspond the hierarchical structure of attribute information to the hierarchical structure of position information. That is, the encoding device (200) can make scalable decoding of attribute information easier. In addition, the encoding device (200) can reduce excessive processing and suppress the increase in the load of encoding processing.

[0452] Flow of Hierarchical Processing

[0453] In this case as well, the attribute information encoding process executed in step S604 of FIG. 44 is executed in the same flow as the flowchart of FIG. 18. However, in step S221 of the attribute information encoding process, the layering process is executed in the following flow. An example of the flow of the layering process in this case will be explained with reference to the flowchart of FIG. 45.

[0454] When layered processing is initiated, the layered processing unit (221) of the layered processing unit (211) sets the current layer (current LoD) to be processed as the lowest layer (LoD) in step S621. That is, the lowest layer (LoD) is set as the processing target.

[0455] In step S622, the point classification unit (231) selects a predicted point based on the list of labels attached in the LoD generation processing of step S602 of FIG. 44. That is, the point classification unit (231) constructs a hierarchical structure similar to location information.

[0456] Each of the processes in steps S623 to S630 is executed in the same way as each of the processes in steps S243 to S250 (Fig. 19).

[0457] By performing the processing of each step in this manner, the hierarchical processing unit (211) can correspond the hierarchical structure of attribute information to the hierarchical structure of location information. That is, the hierarchical processing unit (211) can make scalable decoding of attribute information easier. In addition, the hierarchical processing unit (211) can reduce excessive processing and suppress the increase in the load of hierarchical processing.

[0458] Decoding device

[0459] The configuration of the decoding device (400) is the same as in the example of FIG. 23 in this case. That is, the decoding device (400) of FIG. 23 can decode a bit stream generated by encoding a point cloud by applying the present technology described above in <6. LoDGeneration>. However, in this case, when the location information decoding unit (403) decodes the encoded data of the location information, it performs LoD generation processing (subsampling processing) using a list as described above with reference to FIG. 38 to FIG. 42. That is, the location information decoding unit (403) performs LoD generation processing in the same way as the location information decoding unit (202).

[0460] Then, the location information decoding unit (403) supplies the generated list (label column) to the attribute information decoding unit (404). The attribute information decoding unit (404) decodes the encoded data of the attribute information using the LoD generation processing result (label column). That is, the attribute information decoding unit (404) applies the label column (hierarchical structure of location information) as the hierarchical structure of the attribute information and performs decoding. By doing so, the attribute information decoding unit (404) can perform inverse hierarchical processing without performing LoD generation processing (subsampling processing) and with each attribute information having the same hierarchical structure as the location information.

[0461] Accordingly, the decoding device (400) can realize scalable decoding of attribute information. In addition, the decoding device (400) can reduce excessive processing and suppress an increase in the load of decoding processing of the attribute information encoded data.

[0462] In addition, in this case as well, the attribute information decoding unit (404) has the same configuration as in the example of FIG. 24. Also, the reverse layer processing unit (423) has the same configuration as in the example of FIG. 25.

[0463] <Flow of Decoding Processing>

[0464] An example of the flow of decoding processing in this case is explained with reference to the flowchart in FIG. 46. When decoding processing is initiated, the decoding target LoD depth setting unit (401) of the decoding device (400) sets the LoD depth to be decoded in step S651.

[0465] In step S652, the encoded data extraction unit (402) acquires and maintains a bit stream and extracts encoded data of position information and attribute information from the highest LoD to the LoD depth set in step S651.

[0466] In step S653, the location information decoding unit (403) decodes the encoded data of the location information extracted in step S652 and generates location information (decoding result). At that time, the location information decoding unit (403) performs LoD generation processing using a list as described above with reference to FIGS. 38 to 42 and performs label attachment for each node.

[0467] In step S654, the attribute information decoding unit (404) decodes the encoded data of the attribute information extracted in step S652 and generates attribute information (decoding result). At that time, the attribute information decoding unit (404) decodes the encoded data of the attribute information using the result (label column) of the LoD generation processing performed in step S653. In this case as well, the attribute information decoding processing is performed in the same flow as the flowchart of FIG. 27. However, details of the reverse layering processing performed in step S423 will be described later.

[0468] In step S655, the point cloud generation unit (405) generates a point cloud (decoding result) using the location information (decoding result) generated in step S653 and the attribute information (decoding result) generated in step S654, and outputs the point cloud (decoding result).

[0469] When the processing of step S655 is finished, the decoding process is terminated.

[0470] By performing processing of each step in this manner, the decoding device (400) can correspond the hierarchical structure of attribute information to the hierarchical structure of location information. That is, the decoding device (400) can more easily perform scalable decoding of attribute information. In addition, the decoding device (400) can reduce excessive processing and suppress an increase in the load of decoding processing.

[0471] <Flow of Reverse Hierarchy Processing>

[0472] In this case as well, the attribute information decoding process executed in step S654 of FIG. 46 is executed in the same flow as the flowchart of FIG. 27. However, in step S423 of the attribute information decoding process, the inverse layering process is executed in the following flow. An example of the flow of the inverse layering process in this case will be explained with reference to the flowchart of FIG. 47.

[0473] When reverse layering processing is initiated, the layer-by-layer processing unit (441) of the reverse layering processing unit (423) sets the current layer (current LoD) to be processed as the highest layer (LoD) in step S661. That is, the lowest layer (LoD) is set as the processing target.

[0474] In step S662, the prediction unit (451) derives a predicted value of a prediction point from a reference point based on the list labeled in the LoD generation processing of step S653 of FIG. 46 and the location information of the current LoD resolution.

[0475] For example, the prediction unit (451) identifies a reference point of the current LoD based on a list, that is, by considering the hierarchical structure of attribute information as being the same as the hierarchical structure of location information, and derives a predicted value of the prediction point of the current LoD based on the reference point and the location information of the resolution of the current LoD. The method of deriving this predicted value is arbitrary. For example, the prediction unit (451) may derive the predicted value by the same method as in the case of processing in step S463.

[0476] Each of the processes in steps S663 to S666 is executed in the same way as each of the processes in steps S464 to S467 (Fig. 29).

[0477] By performing the processing of each step in this manner, the reverse hierarchical processing unit (423) can correspond the hierarchical structure of attribute information to the hierarchical structure of location information. That is, the decoding device (400) can perform scalable decoding of attribute information more easily. In addition, the reverse hierarchical processing unit (423) can reduce the processing and suppress the increase in the load of the reverse hierarchical processing.

[0478] <7. Bookkeeping>

[0479] Stratification and Re-stratification Methods

[0480] In the above description, Lifting was used as an example of a method for hierarchizing and de-hierarchizing attribute information; however, this technology can be applied to any technology for hierarchizing attribute information. That is, the method for hierarchizing and de-hierarchizing attribute information may be anything other than Lifting.

[0481] <Control Information>

[0482] In each of the embodiments described above, control information regarding the present technology may be transmitted from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) controlling whether to permit (or prohibit) the application of the present technology described above may be transmitted. Additionally, control information specifying the range for permitting (or prohibiting) the application of the present technology described above (e.g., an upper or lower limit of block size, or both, slice, picture, sequence, component, view, layer, etc.) may be transmitted.

[0483] <Surroundings / Nearby>

[0484] In addition, in this specification, positional relationships such as "nearby" or "surroundings" may include not only spatial positional relationships but also temporal positional relationships.

[0485] Computer

[0486] The aforementioned series of processes may be executed by hardware or by software. When the series of processes is executed by software, a program constituting the software is installed on a computer. Here, the computer includes a computer embedded in dedicated hardware, and, for example, a general-purpose personal computer, which is capable of executing various functions by installing various programs.

[0487] FIG. 48 is a block diagram illustrating an example of a computer hardware configuration that executes the aforementioned series of processes by a program.

[0488] In the computer (900) illustrated in FIG. 48, the CPU (Central Processing Unit) (901), ROM (Read Only Memory) (902), and RAM (Random Access Memory) (903) are connected to each other via a bus (904).

[0489] An input / output interface (910) is also connected to the bus (904). An input unit (911), an output unit (912), a memory unit (913), a communication unit (914), and a drive (915) are connected to the input / output interface (910).

[0490] The input unit (911) consists of, for example, a keyboard, mouse, microphone, touch panel, input terminal, etc. The output unit (912) consists of, for example, a display, speaker, output terminal, etc. The memory unit (913) consists of, for example, a hard disk, RAM disk, non-volatile memory, etc. The communication unit (914) consists of, for example, a network interface. The drive (915) drives removable media (921), such as a magnetic disk, optical disk, optical magnetic disk, or semiconductor memory.

[0491] In a computer configured as described above, the CPU (901) performs the aforementioned series of processes by loading a program stored, for example, in a memory unit (913) into RAM (903) via an input / output interface (910) and a bus (904) and executing it. In addition, data and other items necessary for the CPU (901) to execute various processes are also appropriately stored in RAM (903).

[0492] A program that the computer executes can be applied by writing it to a removable media (921), such as a package media. In that case, the program can be installed in a memory unit (913) through an input / output interface (910) by mounting the removable media (921) to a drive (915).

[0493] Additionally, this program may be provided via a wired or wireless transmission medium, such as a local area network, the internet, or digital satellite broadcasting. In that case, the program can be received by the communication unit (914) and installed in the memory unit (913).

[0494] In addition, this program may be pre-installed in the ROM (902) or memory (913).

[0495] <Applicability of this technology>

[0496] In the above description, the application of this technology to the encoding and decoding of point cloud data has been explained, but this technology is not limited to these examples and can be applied to the encoding and decoding of 3D data of any standard. That is, as long as it does not contradict the aforementioned technology, various processing methods such as encoding and decoding methods, and specifications of various data such as 3D data and metadata, are optional. Furthermore, as long as it does not contradict the aforementioned technology, some of the processing or specifications described above may be omitted.

[0497] In addition, although the encoding device (200) and the decoding device (400) have been described above as examples of the application of the present technology, the present technology can be applied to any configuration.

[0498] For example, the present technology can be applied to various electronic devices, such as transmitters or receivers (e.g., television receivers or mobile phones) in wired broadcasting such as satellite broadcasting or cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices that record images on media such as optical discs, magnetic discs, and flash memory, or play images from such storage media (e.g., hard disk recorders or cameras).

[0499] In addition, the present technology may be implemented as a component of a device, such as a processor as a system LSI (Large Scale Integration), e.g., a video processor; a module using multiple processors, e.g., a video module; a unit using multiple modules, e.g., a video unit; or a set having additional functions added to the unit, e.g., a video set.

[0500] In addition, for example, this technology may also be applied to network systems composed of multiple devices. For example, this technology may be implemented as cloud computing in which multiple devices share and jointly process data through a network. For example, this technology may be implemented in a cloud service that provides video (video) services to any terminal, such as a computer, AV (Audio Visual) device, portable information processing terminal, or IoT (Internet of Things) device.

[0501] In addition, in this specification, the term "system" means a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are housed in the same housing. Accordingly, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are all systems.

[0502] <Fields and Uses Where This Technology Is Applicable>

[0503] Systems, devices, processing units, etc., to which this technology is applied can be used in any field, such as transportation, medical, security, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and natural monitoring. Furthermore, their applications are also arbitrary.

[0504] Others

[0505] In addition, in this specification, "flag" refers to information for identifying multiple states, and includes not only information used to identify two states of true (1) or false (0), but also information capable of identifying three or more states. Accordingly, the values ​​that this "flag" can take may be, for example, two values ​​of 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary and may be 1 bit or multiple bits. Furthermore, since identification information (including flags) is assumed to include not only the identification information in a bit stream, but also differential information of the identification information with respect to some reference information in a bit stream, in this specification, "flag" or "identification information" includes not only that information, but also differential information with respect to the reference information.

[0506] In addition, various information (metadata, etc.) regarding encoded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the encoded data. Here, the term "associated" means, for example, enabling the use (linking) of one data when processing the other data. That is, data associated with one another may be integrated into a single data unit or may be treated as individual data. For example, information associated with encoded data (image) may be transmitted over a different transmission path than that encoded data (image). Also, for example, information associated with encoded data (image) may be recorded on a different recording medium (or a different recording area of ​​the same recording medium) than that encoded data (image). Furthermore, this "associated" may be a part of the data, not the entire data. For example, an image and the information corresponding to that image may be associated with each other in any unit, such as multiple frames, a single frame, or a part within a frame.

[0507] In addition, in this specification, terms such as “synthesize,” “multiplex,” “add,” “integrate,” “include,” “store,” “insert,” “insert,” and “insert” mean integrating multiple things into one, such as integrating encoded data and metadata into one data, and mean one method of “associated” described above.

[0508] Furthermore, the embodiments of the present technology are not limited to the embodiments described above, and various modifications are possible within the scope of not departing from the essence of the present technology.

[0509] For example, the configuration described as a single device (or processing unit) may be divided to form multiple devices (or processing units). Conversely, the configuration described as multiple devices (or processing units) above may be integrated to form a single device (or processing unit). Furthermore, configurations other than those described above may be added to the configuration of each device (or each processing unit). Additionally, if the configuration or operation of the system as a whole is substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0510] In addition, for example, the above-described program may be executed on any device. In that case, the device must have the necessary functions (function blocks, etc.) and be able to obtain the necessary information.

[0511] In addition, for example, each step of a flowchart may be executed by a single device or divided among multiple devices. Also, if a step includes multiple processes, the multiple processes may be executed by a single device or divided among multiple devices. In other words, multiple processes included in a single step may be executed as processes of multiple steps. Conversely, processes described as multiple steps may be integrated and executed as a single step.

[0512] Furthermore, for example, the processing of steps describing a program executed by a computer may be executed chronologically in the order described herein, in parallel, or individually at necessary timings, such as when a call is made. That is, as long as no contradiction arises, the processing of each step may be executed in an order different from the order described above. Additionally, the processing of steps describing this program may be executed in parallel with the processing of other programs, or executed in combination with the processing of other programs.

[0513] Furthermore, for example, multiple technologies related to the present invention may be practiced independently as separate entities, provided that no contradiction arises. Of course, any multiple technologies may also be practiced in combination. For example, all or part of the present invention described in one embodiment may be practiced in combination with all or part of the present invention described in another embodiment. Additionally, any or part or part of any of the aforementioned technologies may be practiced in combination with other technologies not described above.

[0514] In addition, the present technology may also take the following configuration.

[0515] (1) A layering unit is provided that performs layering of attribute information by recursively repeating a process for the reference points, wherein each point of the point cloud is classified into a prediction point, which is a point that leaves a difference between the attribute information and the prediction value, and a reference point, which is a point that is referenced when deriving the prediction value.

[0516] The stratification unit selects the prediction point such that, during stratification, a point also exists in the voxel of one upper layer to which the voxel containing the point of the current layer belongs.

[0517] Information processing device.

[0518] (2) The above-mentioned layering section is,

[0519] The predicted value of the predicted point is derived using the attribute information of the reference point, which is weighted according to the distance between the predicted point and the reference point, and

[0520] Deriving the difference value using the above-derived predicted value

[0521] Information processing device described in (1).

[0522] (3) The layering unit derives the predicted value of the predicted point using the attribute information of the reference point, which is weighted according to the distance based on the location information of the resolution of the lowest layer of the point cloud.

[0523] Information processing device described in (2).

[0524] (4) The layering unit derives the predicted value of the predicted point using the attribute information of the reference point, which is weighted according to the distance based on the location information of the resolution of the current layer of the point cloud.

[0525] Information processing device described in (2).

[0526] (5) The stratification unit classifies all points of the current layer into the prediction points, sets a reference point within the voxel of the upper layer, and derives the prediction value of the prediction points using the attribute information of the reference point.

[0527] Information processing device described in (2).

[0528] (6) A quantization unit further comprising a quantization unit that quantizes the difference value of each point of each layer generated by the layering unit using layer-specific weights.

[0529] Information processing device described in any one of (1) to (5).

[0530] (7) The quantization unit quantizes the difference value using the layer-specific weights, which have larger values ​​as the layer increases.

[0531] Information processing device described in (6).

[0532] (8) The difference value quantized by the above quantization unit and the parameter defining the function used to derive the layer-by-layer weight are further encoded, and the encoding unit is further provided to generate encoded data.

[0533] Information processing device described in (6).

[0534] (9) The difference value quantized by the quantization unit and the layer-specific weight are encoded, and the encoding unit is further provided to generate encoded data.

[0535] Information processing device described in (6).

[0536] (10) For attribute information of a point cloud representing a three-dimensional object, the attribute information is hierarchized by recursively repeating the process of classifying each point of the point cloud into a prediction point, which is a point that leaves a difference between the attribute information and the prediction value, and a reference point, which is a point where the attribute information is referenced when deriving the prediction value, and the attribute information is hierarchized by repeating the process for the reference points.

[0537] In the above layering, the prediction point is selected such that a point also exists in the voxel of the one upper layer to which the voxel containing the point of the current layer belongs.

[0538] Information processing method.

[0539] (11) A reverse layering unit is provided for the attribute information of a point cloud representing a three-dimensional object, which performs reverse layering of the attribute information by recursively repeating the process of classifying each point of the point cloud into a prediction point, which is a point that leaves a difference between the attribute information and the prediction value, and a reference point, which is a point that is referenced when deriving the prediction value, for the reference point.

[0540] The above-mentioned inverse layering unit, during the inverse layering, derives a predicted value of the attribute information of the prediction point using the attribute information of the reference point and the location information of the resolution of a layer other than the lowest layer of the point cloud in each layer, and derives the attribute information of the prediction point using the predicted value and the difference value.

[0541] Information processing device.

[0542] (12) The above-mentioned inverse layering unit derives the predicted value by using the attribute information of the reference point and the location information of the resolution of the current layer of the reference point and the prediction point.

[0543] Information processing device described in (11).

[0544] (13) The above-described inverse layering unit derives the predicted value by using the attribute information of the above-described reference point, the location information of the resolution of one upper layer of the current layer of the above-described reference point, and the location information of the resolution of the above-described current layer of the above-described prediction point.

[0545] Information processing device described in (11).

[0546] (14) The above-mentioned inverse layering unit derives the predicted value of the above-mentioned prediction point using the attribute information of the above-mentioned reference point, which is weighted according to the distance between the above-mentioned prediction point and the above-mentioned reference point.

[0547] Information processing device described in any one of (11) to (13).

[0548] (15) The above-mentioned inverse layering unit layers the attribute information using the above-mentioned location information and corresponds the above-mentioned location information and the attribute information of each point in each layer.

[0549] Information processing device described in any one of (11) to (14).

[0550] (16) The above difference value of each point of each quantized layer is further provided with an inverse quantization unit that inversely quantizes the difference value using layer-specific weights.

[0551] Information processing device described in any one of (11) to (15).

[0552] (17) The above inverse quantization unit uses the above-mentioned layer-specific weights, which have larger values ​​as the layer increases, to inversely quantize the quantized difference value.

[0553] Information processing device described in (16).

[0554] (18) A decoding unit further comprising a decoding unit that decodes encoded data and obtains parameters that define a function used to derive the quantized difference value and the layer-by-layer weight.

[0555] Information processing device described in (16).

[0556] (19) A decoding unit further comprising decoding encoded data and obtaining the quantized difference value and the layer-specific weight

[0557] Information processing device described in (16).

[0558] (20) For attribute information of a point cloud representing a three-dimensional object, the inverse hierarchy of the attribute information is performed by recursively repeating the process of classifying each point of the point cloud into a prediction point, which is a point that leaves a difference between the attribute information and the prediction value, and a reference point, which is a point that is referenced when deriving the prediction value, on the reference point.

[0559] In the above-mentioned inverse layering, for each layer, the predicted value of the attribute information of the prediction point is derived using the attribute information of the reference point and the location information of the resolution of layers other than the lowest layer of the point cloud, and the attribute information of the prediction point is derived using the predicted value and the difference value.

[0560] Information processing method.

[0561] (21) A generating unit that generates information representing the hierarchical structure of location information of a point cloud representing a three-dimensional object, and

[0562] A layering unit that performs layering of attribute information of the point cloud to correspond to the layered structure of the location information based on the information generated by the generation unit.

[0563] An information processing device equipped with

[0564] (22) The above generation unit generates the information for the location information obtained by decoding the encoded data of the location information.

[0565] Information processing device described in (21).

[0566] (23) The above generation unit generates the information by using a list to label each node of the hierarchical structure of the location information.

[0567] Information processing device described in (21) or (22).

[0568] (24) The above generation unit attaches a label of a value representing the layer to each newly appearing node in a hierarchical order from the top layer to the bottom layer of the hierarchical structure of the location information.

[0569] Information processing device described in (23).

[0570] (25) The above generation unit performs label attachment for each newly appearing node in each layer of the hierarchical structure of the location information in a mordantic order.

[0571] Information processing device described in (24).

[0572] (26) The above generation unit, when a new node to which DCM (Direct Coding Mode) is applied appears, performs labeling on the node and labeling on other nodes to which DCM is applied that are divided from the node in the lower layer.

[0573] (24) or (25) information processing device.

[0574] (27) The above generation unit performs labeling for the nodes to which the DCM is applied using a different list than the list used for labeling for the nodes to which the DCM is not applied.

[0575] Information processing device described in (26).

[0576] (28) When points corresponding to each of the plurality of labels overlap, the generation unit keeps one of the plurality of labels and deletes the others.

[0577] (26) or (27) information processing device.

[0578] (29) The layering unit selects a prediction point of each layer of the attribute information to correspond to the layered structure of the location information based on the information.

[0579] Information processing device described in any one of (21) to (28).

[0580] (30) Generate information representing the hierarchical structure of location information of a point cloud representing a three-dimensional object, and

[0581] Based on the generated information, the attribute information is layered to correspond to the hierarchical structure of the location information of the point cloud.

[0582] Information processing method.

[0583] (31) A generating unit that generates information representing the hierarchical structure of location information of a point cloud representing a three-dimensional object, and

[0584] Based on the information generated by the generation unit, the hierarchical structure of the attribute information of the point cloud corresponds to the hierarchical structure of the location information, and the inverse layering unit performs inverse layering of the attribute information.

[0585] An information processing device equipped with

[0586] (32) The above generation unit generates the information for the location information obtained by decoding the encoded data of the location information.

[0587] Information processing device described in (31).

[0588] (33) The above generation unit generates the information by using a list to attach labels to each node of the hierarchical structure of the location information.

[0589] Information processing device described in (31) or (32).

[0590] (34) The above generation unit attaches a label of a value representing the layer to each newly appearing node in a hierarchical order from the top layer to the bottom layer of the hierarchical structure of the location information.

[0591] Information processing device described in (33).

[0592] (35) The above generation unit performs label attachment for each newly appearing node in each layer of the hierarchical structure of the location information in a mordantic order.

[0593] Information processing device described in (34).

[0594] (36) The above generation unit performs labeling on the node when a new node to which DCM (Direct Coding Mode) is applied appears, and labeling on other nodes to which DCM is applied that are divided from the node in the lower layer.

[0595] Information processing device described in (34) or (35).

[0596] (37) The above generation unit performs labeling for the nodes to which the DCM is applied using a different list than the list used for labeling for the nodes to which the DCM is not applied.

[0597] Information processing device described in (36).

[0598] (38) When points corresponding to each of the plurality of labels overlap, the generation unit keeps one of the plurality of labels and deletes the others.

[0599] Information processing device described in (36) or (37).

[0600] (39) The above-mentioned reverse layer selects a prediction point of each layer of the attribute information to correspond to the hierarchical structure of the location information based on the information.

[0601] Information processing device described in any one of (31) to (38).

[0602] (40) Generate information representing the hierarchical structure of location information of a point cloud representing a three-dimensional object, and

[0603] Based on the generated information, the hierarchical structure of the attribute information of the point cloud corresponds to the hierarchical structure of the location information, thereby performing inverse hierarchy of the attribute information.

[0604] Information processing method.

[0605] (41) A layering unit that layers attribute information of the point cloud to correspond to the layered structure of location information of the point cloud representing a three-dimensional object, and

[0606] A quantization unit that quantizes the attribute information layered by the layering unit using layer-specific weights

[0607] An information processing device equipped with

[0608] (42) The quantization unit quantizes the attribute information using the weights which are the ratio of the number of points in the point cloud to the number of points in the layer of the processing target.

[0609] Information processing device described in (41).

[0610] (43) The above quantization unit quantizes the attribute information of the lowest layer using the weight of the value “1”.

[0611] Information processing device described in (42).

[0612] (44) The above quantization unit quantizes the attribute information using the weights formed by the ratio of the number of points in a layer below the layer of the processing target to the number of points in the layer of the processing target.

[0613] Information processing device described in (41).

[0614] (45) The attribute information of the point cloud is layered to correspond to the layered structure of the location information of the point cloud representing the three-dimensional object, and

[0615] Quantizing the above-mentioned layered attribute information using layer-specific weights

[0616] Information processing method.

[0617] (46) An inverse quantization unit that inversely quantizes attribute information of the point cloud, which is layered to correspond to the layered structure of position information of a point cloud representing a three-dimensional object and quantized using layered weights, using said weights, and

[0618] Inverse layering unit that inversely layers the attribute information inversely quantized by the inverse quantization unit

[0619] An information processing device equipped with

[0620] (47) The above inverse quantization unit inversely quantizes the attribute information using the weights which are the ratio of the number of points in the point cloud to the number of points in the layer of the processing target.

[0621] Information processing device described in (46).

[0622] (48) The above inverse quantization unit inversely quantizes the attribute information of the lowest layer using the above weight of value “1”.

[0623] Information processing device described in (47).

[0624] (49) The above inverse quantization unit inversely quantizes the attribute information using the weights, which are the ratio of the number of points in a layer lower than the layer of the processing target to the number of points in the layer of the processing target.

[0625] Information processing device described in (46).

[0626] (50) The above inverse quantization unit acquires the weights used for quantizing the attribute information and uses the acquired weights to inverse quantize the attribute information.

[0627] Information processing device described in any one of (46) to (49).

[0628] (51) The attribute information of the point cloud is layered to correspond to the layered structure of the location information of the point cloud representing the three-dimensional object, and

[0629] Quantizing the above-mentioned layered attribute information using layer-specific weights

[0630] Information processing method. Explanation of the symbols

[0631] 100: Spatial domain 101: Voxels 102: Points 111: Points 112: Points 113: Points 200: Encoding device 201: Position information encoding unit 202: Location information decoding section 203: Point Cloud Generation Section 204: Attribute Information Encoding Section 205: Bitstream generation section 211: Hierarchical processing unit 212: Quantization section 213: Encoding section 221: Hierarchical Processing Unit 231: Point Classification Section 232: Prediction Department 233: Operation unit 234: Update Department 235: Operation unit 400: Decoder 401: Decoding Target LOD Depth Setting Section 402: Encoded Data Extraction Unit 403: Location information decoding section 404: Attribute Information Decoding Section 405: Point Cloud Generation Section 421: Decoding section 422: Inverse Quantumization Unit 423: Reverse Hierarchy Processing Unit 441: Hierarchical Processing Unit 451: Prediction section 452: Operation unit 453: Merge Processor< / lifting> < / octree>

Claims

Claim 1 An information processing device comprising a stratification unit that performs stratification of attribute information by recursively repeating, for reference points, a process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value, with respect to attribute information of a point cloud representing a three-dimensional object, wherein the stratification unit selects the prediction point such that, during the stratification, a point exists in a voxel of one upper layer to which a voxel in which a point of the current layer exists also belongs. Claim 2 In claim 1, the stratification unit derives the predicted value of the predicted point using the attribute information of the reference point weighted according to the distance between the predicted point and the reference point, and derives the difference value using the derived predicted value, an information processing device. Claim 3 In paragraph 2, the layering unit derives the predicted value of the predicted point using the attribute information of the reference point weighted according to the distance based on the location information of the resolution of the lowest layer of the point cloud, an information processing device. Claim 4 In paragraph 2, the layering unit derives the predicted value of the predicted point using the attribute information of the reference point weighted according to the distance based on the position information of the resolution of the current layer of the point cloud, an information processing device. Claim 5 An information processing device according to claim 1, further comprising a quantization unit that quantizes the difference value of each point of each layer generated by the layering unit using layer-specific weights. Claim 6 In paragraph 5, the quantization unit is an information processing device that quantizes the difference value using the layer-specific weights, the higher the layer, the larger the value. Claim 7 An information processing device according to claim 5, further comprising an encoding unit that encodes the difference value quantized by the quantization unit and a parameter defining a function used when deriving the layer-specific weights, and generates encoded data. Claim 8 An information processing device according to claim 5, further comprising an encoding unit that encodes the difference value quantized by the quantization unit and the layer-specific weights, and generates encoded data. Claim 9 Regarding attribute information of a point cloud representing a three-dimensional object, a method for processing information is to perform hierarchy of attribute information by recursively repeating a process for the reference points to classify each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value, and, during the hierarchy, select the prediction points such that a point exists in a voxel of one upper layer to which a voxel containing a point of the current layer belongs. Claim 10 An information processing device comprising: a reverse layering unit that performs reverse layering of the layered attribute information by recursively repeating, for the reference points, a process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point that references the attribute information when deriving the prediction value, with respect to the attribute information of a point cloud representing a three-dimensional object; wherein, during the reverse layering, the reverse layering unit derives the prediction value of the attribute information of the prediction point using the attribute information of the reference point and the position information of the resolution of a layer other than the lowest layer of the point cloud in each layer, and derives the attribute information of the prediction point using the prediction value and the difference value. Claim 11 In item 10, the above-mentioned inverse layering unit derives the above-mentioned prediction value using the above-mentioned attribute information of the above-mentioned reference point and the above-mentioned position information of the resolution of the current layer of the above-mentioned reference point and the above-mentioned prediction point, an information processing device. Claim 12 In item 10, the inverse layering unit derives the predicted value of the predicted point using the attribute information of the reference point, which is weighted according to the distance between the predicted point and the reference point, in an information processing device. Claim 13 In item 10, the above-mentioned reverse layering unit layers the above-mentioned attribute information using the above-mentioned location information, and corresponds the above-mentioned location information of each point in each layer with the above-mentioned attribute information, an information processing device. Claim 14 An information processing device according to claim 10, further comprising an inverse quantization unit that inversely quantizes the difference value of each point of each quantized layer using a layer-specific weight. Claim 15 In claim 14, the above-mentioned inverse quantization unit is an information processing device that inversely quantizes the quantized difference value using the above-mentioned layer-specific weights, the higher the layer, the larger the value. Claim 16 An information processing device according to claim 14, further comprising a decoding unit that decodes encoded data and obtains parameters that define a function used when deriving the quantized difference value and the layer-specific weight. Claim 17 An information processing device according to claim 14, further comprising a decoding unit that decodes encoded data and obtains the quantized difference value and the hierarchical weight. Claim 18 Regarding attribute information of a point cloud representing a three-dimensional object, an information processing method comprises performing inverse layering of the layered attribute information by recursively repeating, for the reference points, a process of classifying each point of the point cloud into a prediction point, which is a point that retains the difference between the attribute information and the prediction value, and a reference point, which is a point where the attribute information is referenced when deriving the prediction value, and, during the inverse layering, deriving the prediction value of the attribute information of the prediction point using the attribute information of the reference point and the position information of the resolution of the layer other than the lowest layer of the point cloud in each layer, and deriving the attribute information of the prediction point using the prediction value and the difference value. Claim 19 delete Claim 20 delete

Images