Decoding device and method

Abstract

To enable a reduction in decoding efficiency to be suppressed.SOLUTION: A reference point is set on the basis of the center of gravity of a point in the case of hierarchizing attribute information by recursively repeating classification between a prediction point for deriving a difference value between the attribute information and a prediction value of the attribute information, and the reference point to be used to derive the prediction value about the attribute information of each point of point cloud representing a three-dimensional shape object as a set of points. This disclosure is applicable to, for example, an information processing device, an image processing device, an encoding device, a decoding device, an electronic apparatus, an information processing method, or a program or the like.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method capable of suppressing a decrease in encoding efficiency. [Background technology]

[0002] Conventionally, methods for encoding 3D data representing a three-dimensional structure such as a point cloud have been considered (see, for example, Non-Patent Document 1). Point cloud data is composed of geometry data (also called position information) and attribute data (also called attribute information) of each point. Therefore, point cloud encoding is performed for both the geometry data and the attribute data. Various methods have been proposed as a method for encoding attribute data. For example, a technique called Lifting has been proposed (see, for example, Non-Patent Document 2). In addition, a method has been proposed that enables attribute data to be decoded in a scalable manner (see, for example, Non-Patent Document 3).

[0003] In such a Lifting Scheme, attribute data is hierarchically organized by recursively repeating a process of setting a point as a reference point or a prediction point for the reference point. According to this hierarchical structure, a predicted value of attribute data of a prediction point is derived using the attribute data of the reference point, and a difference value between the predicted value and the attribute data is encoded. In such hierarchical organization of attribute data, a method has been proposed in which the first point and the last point in Morton order among the candidates for the reference point are alternately selected as the reference point for each layer (see, for example, Non-Patent Document 4). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] R. Mekuria, Student Member IEEE, K. Blom, P. Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video",tcsvt_paper_submitted_february.pdf [Non-Patent Document 2] Khaled Mammou, Alexis Tourapis, Jungsun Kim, Fabrice Robinet, Valery Valentin, Yeping Su, "Lifting Scheme for Lossy Attribute Encoding in TMC1", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42640, April 2018, San Diego, US [Non-Patent Document 3] Ohji Nakagami, Satoru Kuma, "[G-PCC] Spatial scalability support for G-PCC", ISO / IEC JTC1 / SC29 / WG11 MPEG2019 / m47352, March 2019, Geneva, CH [Non-Patent Document 4] Hyejung Hur, Sejin Oh, "[G-PCC][New Proposal] on improved spatial scalable lifting", ISO / IEC JTC1 / SC29 / WG11 MPEG2019 / M51408, October 2019, Geneva, CH Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method described in Non-Patent Document 4 is not always optimal, and other methods have been sought.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress a decrease in coding efficiency. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present technology includes a hierarchical classification unit that classifies attribute information for each point of a point cloud, which represents a three-dimensional object as a collection of points, into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points that are used to derive the predicted value, recursively for the reference points, and the hierarchical classification unit sets the reference points based on the center of gravity of the points.

[0008] An information processing method according to one aspect of the present technology is an information processing method in which, when hierarchizing attribute information for each point in a point cloud that represents a three-dimensional object as a collection of points, the reference points are set based on the center of gravity of the points by recursively repeating classification of the reference points into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points that are used to derive the predicted value.

[0009] An information processing device according to another aspect of the present technology includes a hierarchical classifier that classifies attribute information for each point of a point cloud, which represents an object having a three-dimensional shape as a collection of points, into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points that are used to derive the predicted value, recursively for the reference points, and hierarchizes the attribute information, and the hierarchical classifier sets the reference points based on a distribution pattern of the points.

[0010] An information processing method according to another aspect of the present technology is an information processing method in which, when hierarchizing the attribute information, the reference points are set based on a distribution pattern of points by recursively repeating classification of attribute information for each point of a point cloud that represents a three-dimensional object as a collection of points into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points used for deriving the predicted value.

[0011] According to yet another aspect of the present technology, an information processing device is an information processing device including: a hierarchical classifier that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a collection of points into a prediction point that derives a difference value between the attribute information and a predicted value of the attribute information, and a reference point used to derive the predicted value, recursively repeating the classification of the reference point; and an encoding unit that encodes information related to the setting of the reference point by the hierarchical classifier.

[0012] According to yet another aspect of the present technology, there is provided an information processing method for hierarchizing attribute information for each point of a point cloud, which represents a three-dimensional object as a collection of points, by recursively repeating classification for the reference points into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points that are used to derive the predicted value, thereby hierarchizing the attribute information and encoding information related to the setting of the reference points.

[0013] According to yet another aspect of the present technology, an information processing device includes a hierarchical classifier that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a collection of points into a prediction point that derives a difference value between the attribute information and a predicted value of the attribute information, and a reference point used to derive the predicted value, recursively for the reference point, and the hierarchical classifier selects, as the reference point, a point closer to a center of a bounding box and a point farthest from the center of the bounding box from among the reference point candidates, alternately for each hierarchy.

[0014] An information processing method according to still another aspect of the present technology is an information processing method in which, when hierarchizing the attribute information for each point of a point cloud that represents a three-dimensional object as a collection of points, a classification is performed recursively for the reference points into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points used for deriving the predicted value, and the method alternately selects, for each hierarchical level, a point that is closer to a center of a bounding box and a point that is farthest from the center of the bounding box as the reference point.

[0015] In an information processing device and method according to one aspect of the present technology, when hierarchizing the attribute information for each point in a point cloud that represents a three-dimensional object as a collection of points, the reference points are recursively classified into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points used to derive the predicted value, so that the reference points are set based on the center of gravity of the points when hierarchizing the attribute information.

[0016] In an information processing device and method according to another aspect of the present technology, when hierarchizing the attribute information, the reference points are set based on the distribution pattern of the points by recursively repeating classification of attribute information for each point in a point cloud that represents a three-dimensional object as a collection of points into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points used to derive the predicted value.

[0017] In an information processing device and method according to still another aspect of the present technology, for attribute information for each point in a point cloud that represents a three-dimensional object as a collection of points, the attribute information is hierarchically organized by repeatedly classifying the reference points into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points used to derive the predicted value, with the reference points being recursively classified. Information relating to the setting of the reference points is then encoded.

[0018] In an information processing device and method according to still another aspect of the present technology, for attribute information for each point in a point cloud that represents a three-dimensional object as a collection of points, classification is repeated recursively for the reference points into prediction points that derive a difference value between the attribute information and a predicted value of the attribute information, and reference points used to derive the predicted value, so that when hierarchizing the attribute information, a point closest to the center of a bounding box and a point farthest from the center of the bounding box are alternately selected as the reference point for each hierarchy. [Brief description of the drawings]

[0019] [Figure 1] FIG. 13 is a diagram illustrating an example of a lifting state. [Diagram 2] 11A to 11C are diagrams illustrating an example of a method for setting reference points based on Morton's order. [Diagram 3] 11A to 11C are diagrams illustrating an example of a method for setting reference points based on Morton's order. [Figure 4]11A to 11C are diagrams illustrating an example of a method for setting reference points based on Morton's order. [Diagram 5] 11A to 11C are diagrams illustrating an example of a method for setting reference points based on Morton's order. [Figure 6] FIG. 11 is a diagram illustrating an example of a reference point setting method. [Figure 7] 11A and 11B are diagrams illustrating an example of a method for setting a reference point based on a center of gravity. [Figure 8] FIG. 11 is a diagram illustrating an example of a method for deriving a center of gravity. [Figure 9] FIG. 11 is a diagram illustrating an example of a method for deriving a center of gravity. [Figure 10] FIG. 13 is a diagram showing an example of a region for which the center of gravity is derived; [Figure 11] FIG. 11 is a diagram illustrating an example of a method for selecting points. [Figure 12] FIG. 13 is a diagram illustrating an example of the same conditions. [Figure 13] FIG. 2 is a block diagram showing an example of the main configuration of an encoding device. [Figure 14] 13 is a block diagram showing an example of the main configuration of an attribute information encoding unit. [Figure 15] 2 is a block diagram showing an example of a main configuration of a hierarchical processing unit. FIG. [Figure 16] 11 is a flowchart illustrating an example of the flow of an encoding process. [Figure 17] 13 is a flowchart illustrating an example of the flow of an attribute information encoding process. [Figure 18] 13 is a flowchart illustrating an example of the flow of a hierarchical process. [Figure 19] 13 is a flowchart illustrating an example of the flow of a reference point setting process. [Figure 20] FIG. 2 is a block diagram showing an example of the main configuration of a decoding device. [Figure 21] 13 is a block diagram showing an example of the main configuration of an attribute information decoding unit. FIG. [Figure 22] 13 is a flowchart illustrating an example of the flow of a decoding process. [Diagram 23] 13 is a flowchart illustrating an example of the flow of an attribute information decoding process. [Figure 24] 13 is a flowchart illustrating an example of the flow of a layer inversion process. [Diagram 25] FIG. 13 is a diagram illustrating an example of a table. [Figure 26] 11 is a diagram illustrating an example of a table and signaling. FIG. [Figure 27] 13 is a flowchart illustrating an example of the flow of a reference point setting process. [Figure 28] FIG. 11 is a diagram showing an example of information to be signaled. [Figure 29] FIG. 13 is a diagram showing an example of a signaling target. [Diagram 30] FIG. 13 is a diagram showing an example of information to be signaled. [Diagram 31] FIG. 13 is a diagram showing an example of information to be signaled. [Diagram 32] A diagram showing an example of syntax when signaling fixed-length bit information. [Diagram 33] A diagram showing an example of syntax when signaling fixed-length bit information. [Diagram 34] FIG. 11 is a diagram showing an example of variable-length bit information to be signaled. [Diagram 35] FIG. 11 is a diagram showing an example of variable-length bit information to be signaled. [Diagram 36] A diagram showing an example of syntax for signaling variable-length bit information. [Figure 37] A diagram showing an example of syntax for signaling variable-length bit information. [Figure 38] 13 is a flowchart illustrating an example of the flow of a reference point setting process. [Figure 39] FIG. 13 is a diagram illustrating an example of a search order. [Diagram 40] 13 is a flowchart illustrating an example of the flow of a reference point setting process. [Diagram 41] FIG. 1 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Setting a reference point 2. First embodiment (Method 1) 3. Second embodiment (Method 2) 4. Third embodiment (Method 3) 5. Fourth embodiment (method 4) 6. Notes

[0021] <1. Setting the reference point> <References supporting technical content and technical terms> The scope of disclosure of the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent documents that were publicly known at the time of filing the application.

[0022] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: (mentioned above)

[0023] In other words, the contents described in the above-mentioned non-patent documents and the contents of other documents referenced in the above-mentioned non-patent documents are also used as the basis for determining the support requirements.

[0024] <Point cloud> Previously, there was 3D data such as point clouds, which represent three-dimensional structures using point position information and attribute information, and meshes, which are composed of vertices, edges, and faces and define three-dimensional shapes using polygonal representations.

[0025] For example, in the case of a point cloud, a three-dimensional structure (an object with a three-dimensional shape) is represented as a collection of many points. Point cloud data (also referred to as point cloud data) is composed of position information (also referred to as geometry data) and attribute information (also referred to as attribute data) of each point. The attribute data can include any information. For example, color information, reflectance information, normal information, etc. of each point may be included in the attribute data. In this way, point cloud data has a relatively simple data structure, and by using a sufficiently large number of points, any three-dimensional structure can be represented with sufficient accuracy.

[0026] <Quantization of position information using voxels> Since such point cloud data has a relatively large amount of data, an encoding method using voxels was devised to compress the amount of data by encoding, etc. A voxel is a three-dimensional region for quantizing geometry data (position information).

[0027] That is, a three-dimensional region (also called a bounding box) containing a point cloud is divided into small three-dimensional regions called voxels, and each voxel indicates whether it contains a point or not. In this way, the position of each point is quantized in voxel units. Therefore, by converting point cloud data into such voxel data (also called voxel data), it is possible to suppress an increase in the amount of information (typically, to reduce the amount of information).

[0028] <octree> Furthermore, for geometry data, it was considered to construct an octree using such voxel data. An octree is a tree-structured version of voxel data. The value of each bit in the lowest node of this octree indicates whether or not each voxel has a point. For example, a value of "1" indicates a voxel that contains a point, and a value of "0" indicates a voxel that does not contain a point. In an octree, one node corresponds to eight voxels. In other words, each node in the octree is made up of eight bits of data, and these eight bits indicate whether or not each of the eight voxels has a point.

[0029] The upper nodes of the Octree indicate whether or not there is a point in the area that combines the eight voxels corresponding to the lower nodes belonging to that node. In other words, the upper nodes are generated by combining the information of the voxels of the lower nodes. Note that a node with a value of "0", that is, a node in which none of the corresponding eight voxels contains a point, is deleted.

[0030] In this way, a tree structure (Octree) consisting of nodes whose values ​​are not "0" is constructed. In other words, the Octree can indicate the presence or absence of voxel points at each resolution. By encoding the Octree, the position information can be decoded from the highest resolution (top layer) to the desired layer (resolution) to restore the point cloud data at that resolution. In other words, it is possible to easily decode at any resolution without decoding information at unnecessary layers (resolutions). In other words, it is possible to achieve scalability of voxels (resolutions).

[0031] In addition, by omitting nodes with a value of "0" as described above, it is possible to reduce the resolution of voxels in areas where no points exist, thereby further suppressing the increase in the amount of information (typically reducing the amount of information).

[0032] <lifting> On the other hand, when encoding attribute data (attribute information), the geometry data (position information) is assumed to be known, including degradation due to encoding, and encoding is performed using the positional relationship between points. As a method for encoding such attribute data, a method using a transformation called RAHT (Region Adaptive Hierarchical Transform) or Lifting as described in Non-Patent Document 2 has been considered. By applying these technologies, it is also possible to hierarchically organize attribute data, like the Octree of geometry data.

[0033] For example, in the case of Lifting described in Non-Patent Document 2, attribute data is hierarchically organized by recursively repeating the process of setting a point as a reference point or a prediction point for the reference point. Then, according to this hierarchical structure, a predicted value of the attribute data of the prediction point is derived using the attribute data of the reference point, and the difference value between the predicted value and the attribute data is encoded.

[0034] For example, assume that point P5 is selected as the reference point in Fig. 1. In this case, prediction points are searched for within a circular region of radius R centered on point P5. In this case, point P9 is located within the region, and is therefore set as a prediction point (prediction point with point P5 as the reference point) whose predicted value is derived with reference to point P5.

[0035] By such processing, for example, the difference values ​​of points P7 to P9 indicated by white circles, the difference values ​​of points P1, P3, and P6 indicated by diagonal lines, and the difference values ​​of points P0, P2, P4, and P5 indicated by gray circles are derived as difference values ​​of different hierarchies.

[0036] In reality, the point cloud is arranged in a three-dimensional space and the above-mentioned processing is performed in the three-dimensional space, but for the sake of convenience, the three-dimensional space is shown diagrammatically using a two-dimensional plane in Fig. 1. In other words, the explanation given with reference to Fig. 1 can be similarly applied to processing, phenomena, etc. in the three-dimensional space.

[0037] In the following, explanations of three-dimensional space are appropriately given using a two-dimensional plane, and unless otherwise specified, the explanations can basically be applied to processes and phenomena in three-dimensional space in the same way.

[0038] The reference point in this hierarchy was selected, for example, according to Morton Order. For example, as in the tree structure shown in Figure 2, when selecting one reference point from multiple nodes in a certain hierarchy and making it the node one hierarchy higher, the multiple nodes were searched in Morton Order, and the first node that appeared was selected as the reference point. In Figure 2, each circle represents a node, and a black circle represents the node selected as the reference point (i.e., selected as the node one hierarchy higher). In Figure 2, each node is sorted in Morton Order from left to right. That is, in the example of Figure 2, the leftmost node is always selected.

[0039] In response to this, Non-Patent Document 4 proposes a method for hierarchizing attribute data in such a way that, among the candidates for the reference point, the first point and the last point in Morton's order are alternately selected as the reference point for each layer. That is, as shown in the example of Fig. 3, in the LoD N layer, the first node in Morton's order is selected as the reference point, and in the next layer (LoD N-1), the last node in Morton's order is selected as the reference point.

[0040] FIG. 4 illustrates an example of how a reference point is selected in a three-dimensional space using a two-dimensional plane. Each square in FIG. 4A indicates a voxel in a certain layer. Also, a circle indicates a candidate reference point to be processed. When selecting a reference point from the 2x2 points shown in FIG. 4A, for example, the first point in Morton's order (gray point) is selected as the reference point. In the next higher layer, as shown in FIG. 4B, the last point in Morton's order (gray point) of the 2x2 points is selected as the reference point. Furthermore, in the next higher layer, as shown in FIG. 4C, the first point in Morton's order (gray point) of the 2x2 points is selected as the reference point.

[0041] The arrows shown in each of Fig. 4A to Fig. 4C indicate the movement of the reference point. In this case, the movement range of the reference point is limited to a narrow range as shown in the dotted frame in Fig. 4C, so that the decrease in prediction accuracy is suppressed.

[0042] However, when a reference point is selected at the position shown in Fig. 5 in the same manner as in Fig. 4, the position of the reference point moves as shown in Fig. 5A to Fig. 5C. Fig. 4 explains another example of the selection of a reference point in a three-dimensional space using a two-dimensional plane. In other words, the movement range of the reference point becomes wider than that in Fig. 4, as shown in the dotted frame in Fig. 5C, and there is a risk of reducing prediction accuracy.

[0043] In this way, in the method described in Non-Patent Document 4, there is a risk that prediction accuracy will decrease depending on the position of the point, resulting in a decrease in coding efficiency.

[0044] <How to set a reference point> Therefore, for example, as in method 1 shown in the top row of the table in Fig. 6, in hierarchizing the attribute data, the center of gravity of the points may be obtained and a reference point may be set based on the center of gravity. For example, a point close to the derived center of gravity may be selected as the reference point.

[0045] Also, for example, as in method 2 shown in the second row from the top of the table in FIG. 6, in hierarchizing the attribute data, reference points may be selected according to the distribution pattern (distribution mode) of the points.

[0046] Furthermore, in the hierarchical organization of this attribute data, information regarding the setting of reference points may be transmitted from the encoding side to the decoding side, for example as in method 3 shown in the third row from the top of the table in Figure 6.

[0047] In addition, for example, as in method 4 shown in the fourth row from the top of the table in Figure 6, in hierarchizing this attribute data, points close to the center of the bounding box and points far from it may be selected as reference points alternately for each layer.

[0048] By applying any of these methods, it is possible to suppress a decrease in coding efficiency. Note that the above-mentioned methods can be applied in any combination. Also, the above-mentioned methods can be applied to coding / decoding of attribute data compatible with scalable decoding, and can also be applied to coding / decoding of attribute data not compatible with scalable decoding.

[0049] <2. First embodiment> <Method 1> A case where the above-mentioned "Method 1" is applied will be described. In the case of this "Method 1", the center of gravity of the points is derived, and a reference point is selected based on the center of gravity. Any point relative to the derived center of gravity may be set as the reference point. For example, a point close to the derived center of gravity (for example, a point located closer to the center of gravity) may be selected as the reference point.

[0050] A of FIG. 7 shows an example of a target area for setting a reference point. In A of FIG. 7, a square indicates a voxel, and a circle indicates a point. That is, A of FIG. 7 is a diagram showing an example of a voxel structure in a three-dimensional space, using a two-dimensional plane. For example, if points A to C arranged as shown in A of FIG. 7 are candidates for a reference point, point B close to the center of gravity of these candidates may be selected as a reference point, as shown in B of FIG. 7. B of FIG. 7 shows a hierarchical structure of attribute data, similar to FIG. 2, and the like, and a black circle indicates a reference point. That is, point B is selected as a reference point from points A to C.

[0051] By using a point close to the center of gravity as a reference point in this way, it is possible to set a point close to more other points as a reference point, which means that it is possible to set a reference point so as to suppress a decrease in prediction accuracy for more prediction points, thereby suppressing a decrease in coding efficiency.

[0052] <How to derive the center of gravity> The method of deriving the center of gravity is arbitrary. For example, the center of gravity for any point may be applied as the center of gravity used to select the reference point. For example, the center of gravity of points located within a predetermined range may be derived and used to select the reference point. In this way, it is possible to suppress an increase in the number of points used to derive the center of gravity, and to suppress an increase in the load.

[0053] The range of points used to derive this center of gravity (also referred to as the center of gravity derivation target range) may be any range. For example, the center of gravity of a reference point candidate may be derived as in the method (1) shown in the second row from the top of the table of "center of gravity derivation method" shown in FIG. 8. That is, for example, as shown in FIG. 9A, a voxel region consisting of 2x2x2 voxels in which a point that is a reference point candidate exists may be set as the center of gravity derivation target range. In FIG. 9A, a 2x2x2 voxel region in a three-dimensional space is shown diagrammatically on a two-dimensional plane (as a 2x2 square). In this case, the centers of gravity of the three points indicated by circles in FIG. 9A are derived, and the centers of gravity are used to set the reference point.

[0054] In this way, it is only necessary to derive the center of gravity of the point to be processed (candidate for reference point), and it is therefore not necessary to search for other points, and the center of gravity can be easily derived.

[0055] The voxel region to be the target range for center of gravity derivation is arbitrary and is not limited to 2x2x2. For example, the center of gravity of a point located within a voxel region consisting of NxNxN (N>=2) voxels may be derived. In other words, this NxNxN voxel region may be the target range for center of gravity derivation.

[0056] For example, as shown in A of Fig. 10, the voxel region to be the target range for deriving the center of gravity (the voxel region shown by the thick line in A of Fig. 10) and the target voxel region to derive the reference point (the voxel region shown by the gray in A of Fig. 10) may be located at the same position (the two ranges may be completely coincident). Note that Fig. 10 shows a voxel region that is actually configured in a three-dimensional space, as a schematic representation on a two-dimensional plane. Also, in A of Fig. 10, for convenience of explanation, the target range for deriving the center of gravity and the target voxel region to derive the reference point are shown slightly shifted from each other, but in reality, the two ranges are shown to be completely coincident.

[0057] Also, for example, as shown in B of Fig. 10, the voxel region to be the target range for center of gravity derivation (the voxel region shown by the thick line in B of Fig. 10) may be made wider than the target voxel region to be used for deriving the reference point (the voxel region shown by the gray in B of Fig. 10). In the example of B of Fig. 10, a 4x4x4 voxel region is set as the target range for center of gravity derivation.

[0058] Also, for example, as shown in C of Fig. 10, the center of the voxel region (voxel region indicated by a thick line in C of Fig. 10) that is the target range for the center of gravity derivation may not coincide with the center of the target voxel region (voxel region indicated by a gray in C of Fig. 10) for which the reference point is to be derived. In other words, the target range for the center of gravity derivation may be biased in a predetermined direction with respect to the target voxel region for which the reference point is to be derived. For example, the spread of the target range for the center of gravity derivation may be biased in this way in order to prevent the target range for the center of gravity derivation from going out of the bounding box near the edge of the bounding box.

[0059] Also, for example, the center of gravity of N neighboring points may be obtained as in method (2) shown in the third row from the top of the table of "method of deriving center of gravity" shown in FIG. 8. That is, for example, as shown in B of FIG. 9, N points may be searched from the side closest to the center coordinates of a voxel region consisting of 2x2x2 voxels in which a point that is a candidate for a reference point exists, and the center of gravity of the N points may be derived. In B of FIG. 9, the distribution of points actually arranged in a three-dimensional space is shown on a two-dimensional plane. Also, the black circle indicates the center coordinates of the voxel region from which the reference point is derived. That is, N points (white circles) are selected in order from the side closest to the black circle, and their centers of gravity are derived.

[0060] In this way, the number of points to be searched can be limited to N, and an increase in the load due to the search can be suppressed.

[0061] Also, for example, as in method (3) shown in the fourth row from the top of the table of "methods for deriving center of gravity" shown in Fig. 8, reference point candidates (points existing in a voxel region consisting of 2x2x2 voxels) may be excluded from the neighboring N points derived by method (2). In other words, as shown in C of Fig. 9, the 2x2x2 voxel region may be excluded from the range of interest for deriving the center of gravity, and the center of gravity of points located outside this 2x2x2 voxel region may be derived. In C of Fig. 9, the distribution of points actually arranged in a three-dimensional space is shown diagrammatically on a two-dimensional plane.

[0062] Furthermore, for example, as shown in the fifth row of the table of "method of deriving center of gravity" in FIG. 8, the center of gravity of points within an area of ​​radius r centered on the central coordinates of a voxel area consisting of 2x2x2 voxels in which a point that is a candidate for a reference point exists may be derived. That is, in this case, for example, as shown in D of FIG. 9, the center of gravity of points located within an area of ​​radius r centered on the central coordinates of a voxel area consisting of 2x2x2 voxels, as shown in a dotted frame, is derived. Note that D of FIG. 9 shows a schematic diagram of the distribution of points actually arranged in a three-dimensional space on a two-dimensional plane. Also, a black circle indicates the central coordinates of a voxel area to be derived as a reference point, and a white circle indicates a point within an area of ​​radius r centered on the central coordinates of a voxel area consisting of 2x2x2 voxels.

[0063] By doing this, the technology can also be applied to Lifting, which does not use the voxel structure and is not scalable.

[0064] <How to select a reference point> As described above, in the case of "Method 1," for example, a point close to the derived center of gravity can be set as the reference point. If there are multiple "points close to the center of gravity," one of them is selected as the reference point. This selection method is arbitrary. For example, the reference point may be set as in each method shown in the table of "How to select a reference point from multiple candidates" in FIG. 11.

[0065] For example, as shown in A of Fig. 12, there may be multiple points that are equidistant from the center of gravity. Also, as shown in B of Fig. 12, in order to suppress an increase in the load due to calculations, all points that are located sufficiently close to the center of gravity may be considered to be "points close to the center of gravity." In the case of B of Fig. 12, all points that are located within a radius Dth from the center of gravity are considered to be "points close to the center of gravity." In this case, there may be multiple "points close to the center of gravity."

[0066] In such a case, for example, the point to be processed first in a specified search order may be selected, as shown in method (1) in the second row from the top of the table of "How to select a reference point from multiple candidates" in Figure 11.

[0067] Also, for example, a point to be processed first or last in a predetermined search order may be selected as in method (2) shown in the third row from the top of the table of "Method of selecting a reference point from multiple candidates" shown in Fig. 11. For example, it may be possible to switch for each hierarchical level whether to select the point to be processed first in the predetermined search order or the point to be processed last in the predetermined search order.

[0068] Furthermore, for example, a point to be processed may be selected in the middle (number / 2) of a specified search order, as in method (3) shown in the fourth row from the top of the table of “How to select a reference point from multiple candidates” in FIG. 11.

[0069] Also, for example, as in method (4) shown in the fifth row from the top of the table of "How to select a reference point from multiple candidates" shown in FIG. 11, a point to be processed may be selected in a specified order in a predetermined search order. In other words, a point to be processed Nth in a predetermined search order may be selected. This specified order (N) may be determined in advance, or may be set by a user, an application, etc. Furthermore, when the specified order is settable, information regarding this specified order (N) may be signaled (transmitted).

[0070] As in the above methods (1) to (4), when there are multiple candidates with approximately the same conditions for the center of gravity, a reference point may be set from among the multiple candidates based on a predetermined search order.

[0071] The search order may be any order. For example, it may be Morton's order or may be other than Morton's order. The search order may be predefined by a standard or the like, or may be set by a user, an application, or the like. When the search order is settable, information regarding the search order may be signaled (transmitted).

[0072] Furthermore, for example, as in method (5) shown in the sixth row from the top of the table of "How to select a reference point from multiple candidates" shown in Fig. 11, the range to be subjected to the center of gravity derivation may be made wider, the centers of gravity of points within the wider range to be subjected to the center of gravity derivation may be derived, and a point may be selected using the newly derived centers of gravity. In other words, the conditions may be changed and the centers of gravity may be re-derived.

[0073] <Encoding device> Next, a device to which the present technology is applied will be described. Fig. 13 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology ("Method 1") is applied. The encoding device 100 shown in Fig. 13 is a device that encodes a point cloud (3D data). The encoding device 100 encodes the point cloud by applying the present technology described in the present embodiment.

[0074] Note that Fig. 13 shows the main processing units, data flows, etc., and is not limited to all that is shown in Fig. 13. In other words, in encoding device 100, there may be processing units that are not shown as blocks in Fig. 13, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 13.

[0075] As shown in FIG. 13, the encoding device 100 includes a position information encoding unit 101, a position information decoding unit 102, a point cloud generating unit 103, an attribute information encoding unit 104, and a bit stream generating unit 105.

[0076] The position information encoding unit 101 encodes geometry data (position information) of a point cloud (3D data) input to the encoding device 100. Any encoding method may be used as long as it is compatible with scalable decoding. For example, the position information encoding unit 101 hierarchically organizes the geometry data to generate an octree, and encodes the octree. In addition, for example, filtering and quantization for noise suppression (denoising) may be performed. The position information encoding unit 101 supplies encoded data of the generated geometry data to the position information decoding unit 102 and the bit stream generating unit 105.

[0077] The position information decoding unit 102 obtains the encoded data of the geometry data supplied from the position information encoding unit 101, and decodes the encoded data. Any decoding method may be used as long as it is compatible with the encoding by the position information encoding unit 101. For example, processing such as filtering and inverse quantization for denoising may be performed. The position information decoding unit 102 supplies the generated geometry data (decoded result) to the point cloud generating unit 103.

[0078] The point cloud generating unit 103 acquires attribute data (attribute information) of the point cloud input to the encoding device 100 and geometry data (decoded result) supplied from the position information decoding unit 102. The point cloud generating unit 103 performs processing (recolor processing) to match the attribute data with the geometry data (decoded result). The point cloud generating unit 103 supplies the attribute data associated with the geometry data (decoded result) to the attribute information encoding unit 104.

[0079] The attribute information encoding unit 104 acquires the geometry data (decoded result) and attribute data supplied from the point cloud generating unit 103. The attribute information encoding unit 104 encodes the attribute data using the geometry data (decoded result) to generate encoded data of the attribute data.

[0080] At this time, the attribute information encoding unit 104 applies the above-described present technology (method 1) to encode the attribute data. The attribute information encoding unit 104 supplies the generated encoded data of the attribute data to the bit stream generating unit 105.

[0081] The bitstream generating unit 105 obtains the coded data of the geometry data supplied from the position information encoding unit 101. The bitstream generating unit 105 also obtains the coded data of the attribute data supplied from the attribute information encoding unit 104. The bitstream generating unit 105 generates a bitstream including these coded data. The bitstream generating unit 105 outputs the generated bitstream to the outside of the encoding device 100.

[0082] With this configuration, the encoding device 100 can obtain the center of gravity of points in the hierarchical structure of attribute data, and set a reference point based on the center of gravity. By setting a point close to the center of gravity as the reference point in this way, a point close to more other points can be set as the reference point. Therefore, in other words, a reference point can be set so as to suppress a decrease in prediction accuracy for more prediction points, and a decrease in encoding efficiency can be suppressed.

[0083] Each of these processing units (position information encoding unit 101 to bit stream generating unit 105) of the encoding device 100 has an arbitrary configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.

[0084] <Attribute information encoding part> Fig. 14 is a block diagram showing a main configuration example of the attribute information encoding unit 104 (Fig. 13). Note that Fig. 14 shows main processing units, data flows, etc., and is not limited to all that are shown in Fig. 14. In other words, in the attribute information encoding unit 104, there may be processing units not shown as blocks in Fig. 14, and there may be processing and data flows not shown as arrows, etc. in Fig. 14.

[0085] As shown in FIG. 14, the attribute information encoding unit 104 includes a hierarchical processing unit 111, a quantization unit 112, and an encoding unit 113.

[0086] The layering processing unit 111 performs processing related to layering of attribute data. For example, the layering processing unit 111 acquires attribute data and geometry data (decoded result) supplied from the point cloud generating unit 103. The layering processing unit 111 layer-orders the attribute data using the geometry data. At this time, the layering processing unit 111 performs layering by applying the present technology (method 1) described above. That is, the layering processing unit 111 derives the center of gravity of points in each layer, and selects a reference point based on the center of gravity. Then, the layering processing unit 111 sets a reference relationship in each layer of the hierarchical structure, and derives a predicted value of attribute data of each prediction point using attribute data of the reference point based on the reference relationship, and derives a difference value between the attribute data and the predicted value. The layering processing unit 111 supplies the layered attribute data (difference value) to the quantization unit 112.

[0087] At this time, the layering processing unit 111 may also generate control information related to layering. The layering processing unit 111 may also supply the generated control information to the quantization unit 112 together with the attribute data (difference value).

[0088] The quantization unit 112 acquires the attribute data (difference value) and control information supplied from the hierarchical processing unit 111. The quantization unit 112 quantizes the attribute data (difference value). Any quantization method may be used. The quantization unit 112 supplies the quantized attribute data (difference value) and control information to the encoding unit 113.

[0089] The encoding unit 113 acquires the quantized attribute data (difference value) and control information supplied from the quantization unit 112. The encoding unit 113 encodes the quantized attribute data (difference value) to generate encoded data of the attribute data. Any encoding method may be used. The encoding unit 113 also includes control information in the generated encoded data. In other words, the encoding unit 113 generates encoded data of the attribute data including the control information. The encoding unit 113 supplies the generated encoded data to the bit stream generation unit 105.

[0090] By performing the hierarchical division as described above, the attribute information encoding unit 104 can set a point close to the center of gravity as a reference point, and can therefore set a point close to more other points as a reference point. This means that it is possible to set a reference point so as to suppress a decrease in prediction accuracy for more prediction points, and thus suppress a decrease in encoding efficiency.

[0091] In addition, these processing units (layering processing unit 111 to encoding unit 113) have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.

[0092] <Hierarchical Processing Unit> Fig. 15 is a block diagram showing an example of the main configuration of the hierarchical processing unit 111 (Fig. 14). Note that Fig. 15 shows the main processing units, data flows, etc., and is not limited to all that are shown in Fig. 15. In other words, in the hierarchical processing unit 111, there may be processing units that are not shown as blocks in Fig. 15, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 15.

[0093] As shown in FIG. 15, the layering processing unit 111 has a reference point setting unit 121, a reference relationship setting unit 122, an inversion unit 123, and a weight value derivation unit .

[0094] The reference point setting unit 121 performs processing related to the setting of reference points. For example, the reference point setting unit 121 classifies the points to be processed into reference points whose attribute data are referenced and prediction points from which predicted values ​​of the attribute data are derived, based on the geometry data of each point. That is, the reference point setting unit 121 sets reference points and prediction points. The reference point setting unit 121 repeats this processing recursively for the reference points. That is, the reference point setting unit 121 sets the reference points and prediction points of the layer to be processed, with the reference points set in the previous layer as the processing target. In this way, a hierarchical structure is constructed. That is, the attribute data is hierarchized. The reference point setting unit 121 supplies information indicating the reference points and prediction points of each layer that have been set to the reference relationship setting unit 122.

[0095] The reference relationship setting unit 122 performs processing related to setting of the reference relationship of each layer based on the information supplied from the reference point setting unit 121. That is, for each prediction point of each layer, the reference relationship setting unit 122 sets a reference point (i.e., a reference destination) to be referenced to derive a predicted value of the prediction point. Then, the reference relationship setting unit 122 derives a predicted value of attribute data of each prediction point based on the reference relationship. That is, the reference relationship setting unit 122 derives a predicted value of attribute data of a prediction point using attribute data of a reference point set as a reference destination. Furthermore, the reference relationship setting unit 122 derives a difference value between the derived predicted value and the attribute data of the prediction point. The reference relationship setting unit 122 supplies the derived difference value (hierarchical attribute data) to the inversion unit 123 for each layer.

[0096] The reference point setting unit 121 generates control information and the like related to the hierarchical organization of the attribute data as described above, and supplies it to the quantization unit 112, which can then transmit it to the decoding side.

[0097] The inversion unit 123 performs processing related to inversion of hierarchies. For example, the inversion unit 123 obtains hierarchical attribute data supplied from the reference relationship setting unit 122. In this attribute data, information of each hierarchy is hierarchically arranged in the order of generation. The inversion unit 123 inverts the hierarchy of the attribute data. For example, the inversion unit 123 assigns a hierarchy number (a number for identifying a hierarchy in which the topmost hierarchy is 0, the value is incremented by 1 for each hierarchy level, and the lowest hierarchy is the maximum value) to each hierarchy of the attribute data in the reverse order of the generation order, so that the generation order is from the lowest hierarchy to the top hierarchy. The inversion unit 123 supplies the attribute data with the hierarchy inverted to the weight value derivation unit 124.

[0098] The weight value derivation unit 124 performs processing related to weighting. For example, the weight value derivation unit 124 acquires attribute data supplied from the inversion unit 123. The weight value derivation unit 124 derives a weight value of the acquired attribute data. The weight value derivation unit 124 may derive this weight value by any method. The weight value derivation unit 124 supplies the attribute data (difference value) and the derived weight value to the quantization unit 112 (FIG. 14). The weight value derivation unit 124 may also supply the derived weight value to the quantization unit 112 as control information, and transmit it to the decoding side.

[0099] In the layering processing unit 111 as described above, the reference point setting unit 121 can apply the present technology described above. That is, the reference relationship setting unit 122 can apply the above-mentioned "Method 1" to derive the center of gravity of the points and set the reference point based on the center of gravity. In this way, it is possible to suppress a decrease in prediction accuracy and a decrease in coding efficiency.

[0100] The procedure for this hierarchical division is arbitrary. For example, the process of the reference point setting unit 121 and the process of the reference relationship setting unit 122 may be executed in parallel. For example, for each layer, the reference point setting unit 121 may set a reference point and a prediction point, and the reference relationship setting unit 122 may set a reference relationship.

[0101] In addition, these processing units (reference point setting unit 121 to weight value derivation unit 124) have an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.

[0102] <Encoding process flow> Next, a description will be given of the processing executed by the encoding device 100. The encoding device 100 encodes point cloud data by executing an encoding process. An example of the flow of the encoding process will be described with reference to the flowchart in FIG.

[0103] When the encoding process starts, in step S101, the position information encoding unit 101 of the encoding device 100 encodes the geometry data (position information) of the input point cloud, and generates encoded data of the geometry data.

[0104] In step S102, the position information decoding unit 102 decodes the encoded data of the geometry data generated in step S101 to generate position information.

[0105] In step S103, the point cloud generation unit 103 performs recolor processing using the attribute data (attribute information) of the input point cloud and the geometry data (decoding result) generated in step S102, and makes the attribute data correspond to the geometry data.

[0106] In step S104, the attribute information encoding unit 104 executes an attribute information encoding process to encode the attribute data recolored in step S103, and generate encoded data of the attribute data. At this time, the attribute information encoding unit 104 performs the process by applying the present technology (method 1) described above. For example, the attribute information encoding unit 104 derives the center of gravity of the points in the hierarchical structure of the attribute data, and sets a reference point based on the center of gravity. Details of the attribute information encoding process will be described later.

[0107] In step S105, the bit stream generating unit 105 generates and outputs a bit stream including the coded data of the geometry data generated in step S101 and the coded data of the attribute data generated in step S104.

[0108] When the process of step S105 ends, the encoding process ends.

[0109] By performing the processes at each step in this manner, the encoding device 100 can suppress a decrease in prediction accuracy and suppress a decrease in encoding efficiency.

[0110] <Attribute information encoding process flow> Next, an example of the flow of the attribute information encoding process executed in step S104 in FIG. 16 will be described with reference to the flowchart in FIG.

[0111] When the attribute information encoding process is started, in step S111, the hierarchical processing unit 111 of the attribute information encoding unit 104 performs hierarchical processing to hierarchicalize the attribute data. That is, a reference point and a prediction point of each layer are set, and further, a reference relationship between them is set. At this time, the hierarchical processing unit 111 performs hierarchicalization by applying the above-mentioned present technology (method 1). For example, in hierarchizing the attribute data, the attribute information encoding unit 104 derives a center of gravity of the points, and sets a reference point based on the center of gravity. The hierarchical processing will be described later in detail.

[0112] In step S112, the hierarchical processing unit 111 derives a predicted value of attribute data for each prediction point in each layer of the attribute data hierarchized in step S111, and derives a difference value between the attribute data of the prediction point and its predicted value.

[0113] In step S113, the quantization unit 112 quantizes each of the difference values ​​derived in step S112.

[0114] In step S114, the encoding unit 113 encodes the difference value quantized in step S112 to generate encoded data of the attribute data.

[0115] When the process of step S114 ends, the attribute information encoding process ends, and the process returns to FIG.

[0116] By performing the processing of each step in this manner, the hierarchical processing unit 111 can apply the above-mentioned "Method 1" to derive the center of gravity of points in the hierarchicalization of attribute data and set a reference point based on the center of gravity. Therefore, the hierarchical processing unit 111 can hierarchize the attribute data so as to suppress a decrease in prediction accuracy, thereby suppressing a decrease in encoding efficiency.

[0117] <Flow of layering process> Next, an example of the flow of the layering process executed in step S111 of FIG. 17 will be described with reference to the flowchart of FIG.

[0118] When the layering process is started, in step S121, the reference point setting unit 121 of the layering processing unit 111 sets the value of the variable LoD Index, which indicates the layer to be processed, to an initial value (for example, "0").

[0119] In step S122, the reference point setting unit 121 executes a reference point setting process to set reference points in the hierarchical layer to be processed (that is, to set prediction points as well). The reference point setting process will be described later in detail.

[0120] In step S123, the reference relationship setting unit 122 sets the reference relationship of the hierarchical layer to be processed (which reference point is to be referenced in deriving the predicted value of each prediction point).

[0121] In step S124, the reference point setting unit 121 increments the LoD Index and sets the processing target to the next layer.

[0122] In step S125, the reference point setting unit 121 judges whether all points have been processed. If it is judged that there are unprocessed points, that is, if it is judged that the hierarchical division is not completed, the process returns to step S122 and the subsequent processes are repeated. In this manner, each process from step S122 to step S125 is executed for each hierarchical layer, and if it is judged in step S125 that all points have been processed, the process proceeds to step S126.

[0123] In step S126, the inversion unit 123 inverts the hierarchy of the attribute data generated as described above, and assigns hierarchy numbers to each hierarchy in the reverse order to the generation order.

[0124] In step S127, the weight value derivation unit 124 derives weight values ​​for the attribute data of each hierarchy.

[0125] When the process of step S127 ends, the process returns to FIG.

[0126] By performing the processing of each step in this manner, the hierarchical processing unit 111 can apply the above-mentioned "Method 1" to derive the center of gravity of points in the hierarchicalization of attribute data and set a reference point based on the center of gravity. Therefore, the hierarchical processing unit 111 can hierarchize the attribute data so as to suppress a decrease in prediction accuracy, thereby suppressing a decrease in encoding efficiency.

[0127] <Flow of reference relationship setting process> Next, an example of the flow of the reference point setting process executed in step S122 of FIG. 18 will be described with reference to the flowchart of FIG.

[0128] When the reference point setting process is started, the reference point setting unit 121 identifies a group of points to be used for deriving the center of gravity in step S141, and derives the center of gravity of the group of points to be processed. As described above, the method of deriving this center of gravity is arbitrary. For example, the center of gravity may be derived using any of the methods shown in the table of FIG.

[0129] In step S142, the reference point setting unit 121 selects a point close to the center of gravity derived in step S141 as a reference point. The method of selecting this reference point is arbitrary. For example, the center of gravity may be derived using any of the methods shown in the table of FIG.

[0130] When the process of step S142 ends, the reference point setting process ends, and the process returns to FIG.

[0131] By performing the processing of each step in this manner, the reference point setting unit 121 can apply the above-mentioned "Method 1" to derive the center of gravity of the points in the hierarchicalization of the attribute data, and set the reference point based on the center of gravity. Therefore, the hierarchicalization processing unit 111 can hierarchize the attribute data so as to suppress a decrease in prediction accuracy, thereby suppressing a decrease in encoding efficiency.

[0132] <Decryption device> Next, another example of a device to which the present technology is applied will be described. Fig. 20 is a block diagram showing an example of a configuration of a decoding device, which is one aspect of an information processing device to which the present technology is applied. The decoding device 200 shown in Fig. 20 is a device that decodes encoded data of a point cloud (3D data). The decoding device 200 decodes the encoded data of the point cloud by applying the present technology (method 1) described in the present embodiment.

[0133] Note that Fig. 20 shows the main processing units, data flows, and the like, and is not limited to all that is shown in Fig. 20. In other words, in the decoding device 200, there may be processing units that are not shown as blocks in Fig. 20, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 20.

[0134] As shown in FIG. 20, the decoding device 200 includes an encoded data extraction unit 201, a position information decoding unit 202, an attribute information decoding unit 203, and a point cloud generation unit 204.

[0135] The coded data extraction unit 201 acquires and holds a bit stream input to the decoding device 200. The coded data extraction unit 201 extracts coded data of geometry data (position information) and attribute data (attribute information) from the bit stream it holds. The coded data extraction unit 201 supplies the coded data of the extracted geometry data to the position information decoding unit 202. The coded data extraction unit 201 supplies the coded data of the extracted attribute data to the attribute information decoding unit 203.

[0136] The position information decoding unit 202 acquires the coded data of the geometry data supplied from the coded data extraction unit 201. The position information decoding unit 202 decodes the coded data of the geometry data to generate geometry data (decoded result). This decoding method may be any method similar to that used by the position information decoding unit 102 of the encoding device 100. The position information decoding unit 202 supplies the generated geometry data (decoded result) to the attribute information decoding unit 203 and the point cloud generation unit 204.

[0137] The attribute information decoding unit 203 acquires the encoded data of the attribute data supplied from the encoded data extraction unit 201. The attribute information decoding unit 203 acquires the geometry data (decoded result) supplied from the position information decoding unit 202. The attribute information decoding unit 203 uses the position information (decoded result) to decode the encoded data of the attribute data by a method to which the present technology (method 1) described above is applied, and generates attribute data (decoded result). The attribute information decoding unit 203 supplies the generated attribute data (decoded result) to the point cloud generation unit 204.

[0138] The point cloud generating unit 204 acquires geometry data (decoding result) supplied from the position information decoding unit 202. The point cloud generating unit 204 acquires attribute data (decoding result) supplied from the attribute information decoding unit 203. The point cloud generating unit 204 generates a point cloud (decoding result) using the geometry data (decoding result) and the attribute data (decoding result). The point cloud generating unit 204 outputs the generated point cloud (decoding result) data to the outside of the decoding device 200.

[0139] With this configuration, the decoding device 200 can select a point close to the center of gravity of the points as a reference point in the inverse layering. Therefore, the decoding device 200 can correctly decode the coded data of the attribute data coded by, for example, the above-mentioned coding device 100. Therefore, it is possible to suppress a decrease in prediction accuracy and a decrease in coding efficiency.

[0140] In addition, these processing units (the encoded data extraction unit 201 to the point cloud generation unit 204) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.

[0141] <Attribute information decoding part> Fig. 21 is a block diagram showing a main configuration example of the attribute information decoding unit 203 (Fig. 20). Note that Fig. 21 shows main processing units, data flows, etc., and is not limited to all that are shown in Fig. 21. In other words, in the attribute information decoding unit 203, there may be processing units not shown as blocks in Fig. 21, and there may be processing and data flows not shown as arrows, etc. in Fig. 21.

[0142] As shown in FIG. 21, the attribute information decoding unit 203 has a decoding unit 211, an inverse quantization unit 212, and an inverse layer processing unit 213.

[0143] The decoding unit 211 performs processing related to the decoding of the coded data of the attribute data. For example, the decoding unit 211 obtains the coded data of the attribute data supplied to the attribute information decoding unit 203.

[0144] The decoding unit 211 decodes the coded data of the attribute data to generate attribute data (decoded result). Any decoding method may be used as long as it corresponds to the coding method used by the coding unit 113 (FIG. 14) of the coding device 100. The generated attribute data (decoded result) corresponds to the attribute data before coding, is a difference value between the attribute data and its predicted value, and is quantized. The decoding unit 211 supplies the generated attribute data (decoded result) to the inverse quantization unit 212.

[0145] If the encoded data of the attribute data includes control information related to weight values ​​and control information related to hierarchical organization of the attribute data, the decoding unit 211 also supplies the control information to the inverse quantization unit 212 .

[0146] The inverse quantization unit 212 performs processing related to inverse quantization of attribute data. For example, the inverse quantization unit 212 obtains attribute data (decoding result) and control information supplied from the decoding unit 211.

[0147] The inverse quantization unit 212 inverse quantizes the attribute data (decoding result). At that time, when control information related to weight values ​​is supplied from the decoding unit 211, the inverse quantization unit 212 also acquires the control information, and performs inverse quantization of the attribute data (decoding result) based on the control information (using weight values ​​derived based on the control information).

[0148] Furthermore, when control information related to hierarchical organization of attribute data is supplied from the decoding unit 211, the inverse quantization unit 212 also acquires the control information.

[0149] The inverse quantization unit 212 supplies the inverse quantized attribute data (decoding result) to the inverse hierarchical processing unit 213. Furthermore, when control information related to hierarchical processing of the attribute data is acquired from the decoding unit 211, the inverse quantization unit 212 also supplies the control information to the inverse hierarchical processing unit 213.

[0150] The inverse hierarchical processing unit 213 obtains the inversely quantized attribute data (decoding result) supplied from the inverse quantization unit 212. As described above, this attribute data is a difference value. The inverse hierarchical processing unit 213 also obtains the geometry data (decoding result) supplied from the position information decoding unit 202. The inverse hierarchical processing unit 213 uses the geometry data to perform inverse hierarchical processing, which is the inverse process of the hierarchical processing performed by the hierarchical processing unit 111 (FIG. 14) of the encoding device 100, on the obtained attribute data (difference value).

[0151] Here, the inverse hierarchical structure will be described. For example, the inverse hierarchical structure processor 213 performs hierarchical structure processing on the attribute data using a method similar to that of the encoding device 100 (hierarchical structure processor 111) based on the geometry data supplied from the position information decoder 202. That is, the inverse hierarchical structure processor 213 sets reference points and prediction points for each layer based on the decoded geometry data, and sets a hierarchical structure for the attribute data. The inverse hierarchical structure processor 213 further sets a reference relationship (a reference destination for each prediction point) for each layer of the hierarchical structure using the reference points and prediction points.

[0152] The inverse layer processing unit 213 then inversely layers the acquired attribute data (difference values) using the hierarchical structure and the reference relationships between the layers. That is, the inverse layer processing unit 213 derives a prediction value of a prediction point from a reference point according to the reference relationships, and restores attribute data of each prediction point by adding the prediction value to the difference value. The inverse layer processing unit 213 performs this process for each layer from a higher layer toward a lower layer. That is, the inverse layer processing unit 213 uses a prediction point from which attribute data has been restored in a layer higher than the layer to be processed as a reference point, and restores attribute data of a prediction point in the layer to be processed as described above.

[0153] In the de-hierarchization performed in this manner, the de-hierarchization processor 213 applies the above-described present technology (method 1) to set reference points when layering attribute data based on decoded geometry data. That is, the de-hierarchization processor 213 derives the center of gravity of the points and selects a point close to the center of gravity as a reference point. The de-hierarchization processor 213 supplies the de-hierarchized attribute data to the point cloud generator 204 (FIG. 20) as a decoding result.

[0154] By performing inverse hierarchical construction as described above, the inverse hierarchical construction processor 213 can set a point close to the center of gravity as a reference point, and can therefore perform hierarchical construction of attribute data so as to suppress a decrease in prediction accuracy. That is, the attribute information decoding unit 203 can correctly decode coded data coded by a similar method. For example, the attribute information decoding unit 203 can correctly decode coded data of attribute data coded by the above-mentioned attribute information coding unit 104. Therefore, a decrease in coding efficiency can be suppressed.

[0155] In addition, these processing units (the decoding unit 211 to the reverse layering processing unit 213) may have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.

[0156] <Decryption process flow> Next, a description will be given of the processing executed by the decoding device 200. The decoding device 200 decodes the encoded data of the point cloud by executing a decoding process. An example of the flow of this decoding process will be described with reference to the flowchart in FIG.

[0157] When the decoding process is started, in step S201, the coded data extraction unit 201 of the decoding device 200 acquires and holds a bit stream, and extracts coded data of geometry data and coded data of attribute data from the bit stream.

[0158] In step S202, the position information decoding unit 202 decodes the encoded data of the extracted geometry data to generate geometry data (decoded result).

[0159] In step S203, the attribute information decoding unit 203 executes attribute information decoding processing, decodes the coded data of the attribute data extracted in step S201, and generates attribute data (decoded result). At this time, the attribute information decoding unit 203 performs processing by applying the present technology (method 1) described above. For example, the attribute information decoding unit 203 derives the center of gravity of the points in the hierarchical structure of the attribute data, and sets a point close to the center of gravity as a reference point. Details of the attribute information decoding processing will be described later.

[0160] In step S204, the point cloud generating unit 204 generates and outputs a point cloud (decoding result) using the geometry data (decoding result) generated in step S202 and the attribute data (decoding result) generated in step S203.

[0161] When the process of step S204 ends, the decoding process ends.

[0162] By performing the processes of each step in this manner, the decoding device 200 can correctly decode the coded data of the attribute data coded by the same method. For example, the decoding device 200 can correctly decode the coded data of the attribute data coded by the coding device 100 described above. Therefore, it is possible to suppress a decrease in prediction accuracy and a decrease in coding efficiency.

[0163] <Flow of attribute information decryption process> Next, an example of the flow of the attribute information decoding process executed in step S203 in FIG. 22 will be described with reference to the flowchart in FIG.

[0164] When the attribute information decoding process is started, in step S211, the decoding unit 211 of the attribute information decoding unit 203 decodes the coded data of the attribute data to generate attribute data (decoded result). This attribute data (decoded result) is quantized as described above.

[0165] In step S212, the inverse quantization unit 212 executes an inverse quantization process to inverse quantize the attribute data (decoding result) generated in step S211.

[0166] In step S213, the inverse hierarchical processing unit 213 executes an inverse hierarchical processing to inversely hierarchicalize the attribute data (difference value) inversely quantized in step S212, and derive the attribute data of each point. At this time, the inverse hierarchical processing unit 213 performs the inverse hierarchicalization by applying the present technology (method 1) described above. For example, in hierarchizing the attribute data, the inverse hierarchical processing unit 213 derives the center of gravity of the points, and sets a point close to the center of gravity as a reference point. The inverse hierarchical processing will be described later in detail.

[0167] When the process of step S213 ends, the attribute information decoding process ends, and the process returns to FIG.

[0168] By performing the processing of each step in this manner, the attribute information decoding unit 203 can apply the above-mentioned "Method 1" and set a point close to the center of gravity of the points as a reference point in the hierarchicalization of the attribute data. Therefore, the inverse hierarchicalization processing unit 213 can hierarchically organize the attribute data so as to suppress a decrease in prediction accuracy. That is, the attribute information decoding unit 203 can correctly decode the encoded data encoded by the same method. For example, the attribute information decoding unit 203 can correctly decode the encoded data of the attribute data encoded by the above-mentioned attribute information encoding unit 104. Therefore, a decrease in encoding efficiency can be suppressed.

[0169] <Flow of reverse hierarchy processing> Next, an example of the flow of the hierarchical inverse process executed in step S213 in FIG. 23 will be described with reference to the flowchart in FIG.

[0170] When the inverse layering process is started, in step S221, the inverse layering processor 213 performs layering process on the attribute data (decoded result) using the geometry data (decoded result), restores the reference points and prediction points of each layer set on the encoding side, and further restores the reference relationship of each layer. That is, the inverse layering processor 213 performs the same process as the layering process performed by the layering processor 111, sets the reference points and prediction points of each layer, and further sets the reference relationship of each layer.

[0171] For example, like the layering processor 111, the inverse layering processor 213 applies the above-mentioned "Method 1" to derive the center of gravity of the points, and sets a point close to the center of gravity as a reference point.

[0172] In step S222, the inverse layer processing unit 213 performs inverse layering of the attribute data (decoding result) using this hierarchical structure and reference relationship, and restores the attribute data of each point. That is, the inverse layer processing unit 213 derives a predicted value of the attribute data of the prediction point from the attribute data of the reference point based on the reference relationship, and adds the predicted value to the difference value of the attribute data (decoding result) to restore the attribute data.

[0173] When the process of step S222 ends, the layer inverse process ends, and the process returns to FIG.

[0174] By performing the process of each step in this manner, the inverse layering processor 213 can realize layering similar to that achieved during encoding. That is, the attribute information decoder 203 can correctly decode encoded data encoded using a similar method. For example, the attribute information decoder 203 can correctly decode the encoded data of attribute data encoded by the above-described attribute information encoder 104. This makes it possible to suppress a decrease in encoding efficiency.

[0175] <3. Second embodiment> <Method 2> Next, a case where the above-mentioned "Method 2" is applied will be described with reference to Fig. 6. In the case of this "Method 2", in the hierarchical organization of attribute data, a reference point is selected according to the distribution pattern (distribution mode) of the points.

[0176] For example, in the table (table information) shown in Fig. 25, the distribution pattern (distribution mode) of points in the processing target region in which the reference point is set is associated with information (index) indicating the point to be selected in that case. For example, the second row from the top of this table shows that when the distribution pattern of points in the 2x2x2 voxel region in which the reference point is set is "10100001", the point with index "2", that is, the point that appears second, is selected. Each bit value of the distribution pattern "10100001" indicates the presence or absence of a point in each 2x2x2 voxel, with a value of "1" indicating that a point exists in the voxel to which that bit is assigned, and a value of "0" indicating that a point does not exist in the voxel to which that bit is assigned.

[0177] Similarly, in this table, the index of the point to be selected is shown for each distribution pattern. In other words, in the hierarchical organization of attribute data, this table is referenced and a point with an index corresponding to the distribution pattern of points in the processing target area for which the reference point is to be set is selected as the reference point.

[0178] In this way, the reference points can be selected more easily.

[0179] This table information may be any information that links the distribution pattern of points with information indicating the points to be selected. For example, it may be table information that selects points close to the center of gravity for each distribution pattern of points, as in method (1) shown in the second row from the top of the "table" shown in A of Fig. 26. In other words, each distribution pattern may be linked to an index of a point close to the center of gravity for that distribution pattern.

[0180] Also, for example, as in method (2) shown in the third row from the top of the "table" shown in A of Fig. 26, it may be table information that selects arbitrary points for each distribution pattern of points. Furthermore, for example, as in method (3) shown in the fourth row from the top of this "table", it may be possible to select a table to be used from among multiple tables. For example, it may be possible to switch the table to be used depending on the hierarchy (depth of LoD).

[0181] This table information may be prepared in advance. For example, predetermined table information may be specified by a standard. In this case, signaling (transmission from the encoding side to the decoding side) of the table information is not required.

[0182] Moreover, this table information may be derived from geometry data by the position information decoding unit 102. Moreover, the position information decoding unit 202 may also derive this table information from geometry data (decoding result). In this case, signaling (transmission from the encoding side to the decoding side) of the table information is not necessary.

[0183] Of course, this table information may be generated (or updated) by a user, an application, etc. In that case, the generated (or updated) table information may be signaled. That is, for example, the encoding unit 113 may encode information related to this table information, and the encoded data may be included in a bit stream, for example, to signal the information.

[0184] Also, as described above, this table information may be switched depending on the hierarchy (depth of LoD). In that case, as shown in the second row from the top of the "Table" table in FIG. 26B, the switching method may be specified in advance in a standard or the like, and information indicating the switching method may not be signaled.

[0185] Also, an index (identification information) indicating the selected table may be signaled, as in method (2) shown in the third row from the top of the "Table" table shown in B of Fig. 26. For example, this index may be signaled in an attribute parameter set (AttributeParameterSet).

[0186] Furthermore, the selected table information itself may be signaled, for example, as in method (3) shown in the fourth row from the top of the "Table" table shown in B of Fig. 26. For example, this table information may be signaled in an attribute brick header.

[0187] Also, as in method (4) shown in the fifth row from the top of the "Table" table shown in FIG. 26B, a part of the selected table information may be signaled. In other words, it may be possible to partially update the table information. For example, this table information may be signaled in an attribute brick header.

[0188] When this method 2 is applied, the configurations of the encoding device 100 and the decoding device 200 are basically the same as when the above-mentioned method 1 is applied. Therefore, the encoding device 100 can execute each process, such as the encoding process, the attribute information encoding process, and the hierarchical process, in the same flow as in the first embodiment.

[0189] <Reference point setting process flow> An example of the flow of the reference point setting process in this case will be described with reference to the flowchart in Fig. 27. When the reference point setting process starts, in step S301, the reference point setting unit 121 refers to table information and selects reference points according to the point distribution pattern.

[0190] In step S302, the reference point setting unit 121 determines whether or not to signal information on the used table. If it is determined that information is to be signaled, the process proceeds to step S303.

[0191] In step S303, the reference point setting unit 121 signals information about the used table. When the process of step S303 ends, the reference point setting process ends, and the process returns to FIG.

[0192] By transmitting the table information from the encoding device 100 in this manner, the decoding device 200 can perform decoding using the table information.

[0193] The decoding device 200 can execute each process, such as the decoding process, the attribute information decoding process, and the inverse layering process, in the same manner as in the first embodiment.

[0194] <4. Third embodiment> <Method 3> Next, a case where the above-mentioned "method 3" is applied will be described with reference to Fig. 6. In the case of this "method 3", a reference point set in the hierarchical structure of attribute data may be signaled.

[0195] For example, as in method (1) shown in the second row from the top of the table of "Signaling Target" in Fig. 28, information indicating whether to refer to all nodes (all points), i.e., whether to set them as reference points or prediction points, may be signaled. For example, as shown in A of Fig. 29, all nodes in all hierarchical levels may be sorted in Morton order, and indexes (index 0 to index K) may be assigned to each node. In other words, each node (and information assigned to each node) can be identified by index 0 to index K.

[0196] Also, for example, as in method (2) shown in the third row from the top of the "Signaling Target" table in Fig. 28, information indicating which node (point) is to be referenced and selected for some hierarchical layers may be signaled. For example, as shown in B in Fig. 29, a layer (LoD) to be the target of signaling may be specified, all nodes in that layer may be sorted in Morton order, and an index may be assigned to each node.

[0197] For example, suppose that the attribute data has a hierarchical structure as shown in A of Fig. 30. That is, one each of point #0 of LoD2, point #1 of LoD2, and point #2 of LoD2 are selected as reference points to form each point #0 of LoD1. In this case, if indexes are assigned to the points of LoD2 in the search order as shown in B of Fig. 30, each point of LoD1#0 is represented by the index of LoD2 as shown in C of Fig. 30. In other words, the distribution pattern of points of LoD1#0 can be represented by specifying "LoD2 0,1,0".

[0198] In this way, the index of LoD N-1 can specify a 2x2x2 voxel region of LoD N. That is, one 2x2x2 voxel can be specified by the LoD (layer specification) and order (m-th). By specifying by layer and index in this way, signaling can be performed for only some layers as necessary, so that the increase in the amount of code can be suppressed compared to the case of method (1), and the decrease in coding efficiency can be suppressed.

[0199] Furthermore, for example, as in method (3) shown in the fourth row from the top of the "Signaling Target" table shown in FIG. 28, the points to be signaled may be limited by the number of points in the lower NxNxN voxel region. In other words, information regarding the setting of reference points for points that satisfy a predetermined condition may be signaled. In this way, as shown in C of FIG. 29, it is possible to further reduce the nodes to be signaled. This makes it possible to suppress a decrease in coding efficiency.

[0200] For example, suppose that the attribute data has a hierarchical structure as shown in A of FIG. 31. In this case, suppose that the target of signaling is limited to a 2x2x2 voxel region including three or more points. If indices are assigned to the points of LoD2 in the search order as shown in B of FIG. 31, the voxel region shown on the right side of LoD2 is excluded from the target of signaling. Therefore, no index is assigned to this voxel region. Therefore, as shown in C of FIG. 31, the amount of signaling data can be reduced more than in C of FIG. 30, and an increase in the amount of code can be suppressed.

[0201] In addition, for example, method (2) and method (3) may be applied in combination, such as method (4) shown in the fifth row from the top of the “Signaling Target” table shown in FIG.

[0202] <Fixed-length signaling> The above signaling may be performed using fixed-length data. For example, signaling may be performed using the syntax shown in A of FIG. 32. In the syntax in A of FIG. 32, num_Lod is a parameter indicating the number of LoDs to be signaled. lodNo[ i ] is a parameter indicating the Lod number. voxelType[i] is a parameter indicating the type of voxel to be signaled. By specifying this parameter, it is possible to limit the transmission target within the Lod. num_node is a parameter indicating the actual number of signals. This num_node can be derived from the geometry data. node[k] indicates the information to be signaled for each 2x2x2 voxel region. k indicates the number of the node in Morton order.

[0203] If parsing is required before obtaining geometry data for parallel processing or the like, signaling may be performed using the syntax shown in B of Fig. 32. If parsing is required in this way, num_node can be signaled.

[0204] Also, when signaling is performed with fixed-length data, the syntax shown in A of Fig. 33 may be applied. In this case, a flag Flag[k] that controls the signal is signaled. In this case, too, when parsing is required before geometry data is obtained for parallel processing or the like, signaling may be performed with the syntax shown in B of Fig. 33. When parsing is required in this way, num_node may be signaled.

[0205] <Variable length signaling> Moreover, the above-mentioned signaling may be performed by variable-length data. For example, as in the example of Fig. 34, the position of a node in a 2x2x2 voxel region may be signaled. In this case, the bit length of the signaling may be set according to the number of nodes in the 2x2x2 voxel region, for example, based on the table information shown in A of Fig. 34.

[0206] On the decoding side, the number of nodes contained in a 2x2x2 voxel region can be obtained from the geometry data. Therefore, even if a bit string such as "10111010..." is input as shown in the example of B in Fig. 34, the number of nodes contained in the 2x2x2 voxel region can be grasped from the geometry data, so that it is possible to divide it into appropriate bit lengths and correctly obtain information for each voxel region.

[0207] Also, as in the example of Fig. 35, an index of the table information to be used may be signaled. In that case, the bit length of the signaling may be made variable according to the number of nodes in a 2x2x2 voxel region, for example, as shown in A of Fig. 35.

[0208] For example, based on the table information shown in A of FIG. 35, if the number of nodes is between 5 and 8, 2 bits are assigned and the table information in B of FIG. 35 is selected. In this case, a bit string "00" indicates that the first node in a predetermined search order (e.g., Morton's order) is selected. Also, a bit string "01" indicates that the first node in the reverse order (reverse) of the predetermined search order (e.g., Morton's order) is selected. Furthermore, a bit string "10" indicates that the second node in the predetermined search order (No reversal) is selected. Also, a bit string "11" indicates that the second node in the reverse order (reversal) of the predetermined search order is selected.

[0209] Also, for example, based on the table information shown in A of Figure 35, if the number of nodes is 3 or 4, 1 bit is assigned and the table information of C of Figure 35 is selected. In this case, a bit string of "0" indicates that the first node in a predetermined search order (e.g., Morton's order) is selected. Also, a bit string of "1" indicates that the first node in the reverse order of the predetermined search order (e.g., Morton's order) is selected.

[0210] On the decoding side, the number of nodes contained in a 2x2x2 voxel region can be obtained from the geometry data. Therefore, even if a bit string such as "10111010..." is input as shown in the example of D in Fig. 35, the number of nodes contained in the 2x2x2 voxel region can be grasped from the geometry data, so that it is possible to divide it into appropriate bit lengths and correctly obtain information for each voxel region.

[0211] An example of the syntax in the case of variable length is shown in Fig. 36. In the syntax in Fig. 36, bitLength is a parameter indicating the bit length, and signalType[i] is a parameter indicating the variable length coding method for each LoD.

[0212] In addition, even in the case of this variable length, if parsing is required before the geometry data is obtained for parallel processing, etc., it is possible to signal num_node or flag[j] as in the syntax shown in Figure 37.

[0213] <Reference point setting process flow> An example of the flow of the reference point setting process in this case will be described with reference to the flowchart in Fig. 38. When the reference point setting process starts, the reference point setting unit 121 selects a reference point in step S321.

[0214] In step S322, reference point setting unit 121 signals information about the reference point set in step S321. When the process of step S322 ends, the reference point setting process ends, and the process returns to Fig. 18. By transmitting the table information from encoding device 100 in this manner, decoding device 200 can perform decoding using the table information.

[0215] <5. Fourth embodiment> <Method 4> Next, a case where the above-mentioned "Method 4" is applied will be described with reference to Fig. 6. In the case of this "Method 4", as the reference point, among the reference point candidates, a point closer to the center of the bounding box and a point farther from the center of the bounding box may be alternately selected for each layer.

[0216] For example, when selecting reference points at the positions shown in Fig. 39, points closer to the center of the bounding box and points farther from the center of the bounding box are alternately selected for each layer, as shown in A to C of Fig. 39. In this way, the movement range of the reference points is limited to a narrow range as shown in the dotted frame, as shown in C of Fig. 39, thereby suppressing a decrease in prediction accuracy.

[0217] By using the center of the bounding box as a reference for the selection direction of points in this way, it is possible to suppress a decrease in prediction accuracy, regardless of the position within the bounding box.

[0218] Note that such point selection may be achieved by changing the search order of points depending on the position within the bounding box. For example, the search order may be the order of distance from the center of the bounding box. In this way, the search order can be changed depending on the position within the bounding box. Also, for example, the search order may be changed for each of the eight divided regions obtained by dividing the bounding box into eight (two in each of the x, y and z directions).

[0219] <Reference point setting process flow> An example of the flow of the reference point setting process in this case will be described with reference to the flowchart in Fig. 40. When the reference point setting process starts, in step S341, the reference point setting unit 121 determines whether or not a point closer to the center of the bounding box has been selected as the reference point of the previous layer.

[0220] If it is determined that the closer point has been selected, the process proceeds to step S342.

[0221] In step S342, the reference point setting unit 121 selects, from among the reference point candidates, the point farthest from the center of the bounding box as the reference point. When the process of step S342 ends, the reference point setting process ends, and the process returns to FIG.

[0222] If it is determined in step S341 that the closer point has not been selected, the process proceeds to step S343.

[0223] In step S343, the reference point setting unit 121 selects, from among the reference point candidates, the point closest to the center of the bounding box as the reference point. When the process of step S343 ends, the reference point setting process ends, and the process returns to FIG.

[0224] By selecting reference points in this manner, the encoding device 100 can suppress a decrease in prediction accuracy of the reference points, thereby suppressing a decrease in encoding efficiency.

[0225] <6. Notes> <Hierarchization / reverse hierarchy method> In the above, Lifting has been described as an example of a method for hierarchizing and dehierarchizing attribute information, but this technology can be applied to any technology for hierarchizing attribute information. In other words, the method for hierarchizing and dehierarchizing attribute information may be other than Lifting. In addition, the method for hierarchizing and dehierarchizing attribute information may be a scalable method such as that described in Non-Patent Document 3, or a non-scalable method.

[0226] <Control information> Control information related to the present technology described in each of the above embodiments may be transmitted from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) that controls whether or not to permit (or prohibit) application of the present technology described above may be transmitted. In addition, for example, control information that specifies a range (e.g., upper or lower limit of block size, or both, slice, picture, sequence, component, view, layer, etc.) in which application of the present technology described above is permitted (or prohibited) may be transmitted.

[0227] <Surroundings / nearby> In this specification, positional relationships such as "nearby" and "around" may include not only spatial positional relationships but also temporal positional relationships.

[0228] <Computer> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, etc., capable of executing various functions by installing various programs.

[0229] FIG. 41 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0230] In a computer 900 shown in FIG. 41, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0231] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0232] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0233] In the computer configured as above, the series of processes described above are performed by the CPU 901, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data and the like necessary for the CPU 901 to execute various processes as appropriate.

[0234] The program executed by the computer can be applied by recording it on a removable medium 921 such as a package medium, for example. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by mounting the removable medium 921 on the drive 915.

[0235] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0236] In addition, this program can also be installed in advance in the ROM 902 or the storage unit 913.

[0237] <Target of application of this technology> Although the above describes the application of this technology to the encoding and decoding of point cloud data, this technology is not limited to these examples and can be applied to the encoding and decoding of 3D data of any standard. In other words, as long as it does not conflict with the above-mentioned technology, various processes such as encoding and decoding methods and specifications of various data such as 3D data and metadata are arbitrary. In addition, as long as it does not conflict with the present technology, some of the above-mentioned processes and specifications may be omitted.

[0238] Furthermore, although the encoding device 100 and the decoding device 200 have been described above as application examples of the present technology, the present technology can be applied to any configuration.

[0239] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) in satellite broadcasting, wired broadcasting such as cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media.

[0240] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0241] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and cooperate with each other to perform processing via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (video images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0242] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.

[0243] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications are also arbitrary.

[0244] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Thus, the value that this "flag" can take may be, for example, a binary value of 1 / 0, or may be three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, it is assumed that identification information (including flags) is not only included in the bit stream, but also includes difference information of the identification information with respect to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information with respect to the reference information.

[0245] Also, various information (metadata, etc.) related to the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, data associated with each other may be combined into one piece of data, or each piece of data may be made into separate data. For example, information associated with coded data (image) may be transmitted on a transmission path different from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium different from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in any unit, such as multiple frames, one frame, or a part of a frame.

[0246] In this specification, the terms "combine," "multiplex," "append," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0247] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0248] For example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0249] Also, for example, the above-mentioned program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0250] Also, for example, each step of one flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0251] Also, for example, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel, or individually at a required timing such as when a call is made. In other words, the processing of each step may be executed in an order different from the order described above, so long as no contradiction occurs. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0252] Also, for example, a plurality of techniques related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any of a plurality of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in other embodiments. Also, a part or all of any of the above-mentioned present technologies can be implemented in combination with other technologies not described above.

[0253] The present technology can also be configured as follows. (1) A hierarchical classifier that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into a prediction point that derives a difference value between the attribute information and a prediction value of the attribute information and a reference point that is used to derive the prediction value, by recursively repeating the classification of the reference points, The hierarchical division unit sets the reference point based on the center of gravity of the points. Information processing device. (2) The hierarchical generation unit sets a point that is closer to the center of gravity among the candidate points as the reference point. An information processing device as described in (1). (3) The hierarchical unit sets the reference point based on a center of gravity of points located within a predetermined range. An information processing device according to (1) or (2). (4) The hierarchical division unit sets the reference point based on a predetermined search order from among a plurality of candidates having substantially the same conditions with respect to the center of gravity. An information processing device according to any one of (1) to (3). (5) For attribute information for each point of a point cloud that represents a three-dimensional object as a set of points, a classification is performed recursively for the reference points into prediction points for deriving a difference value between the attribute information and a prediction value of the attribute information, and reference points used for deriving the prediction value, and the reference points are set based on the center of gravity of the points when classifying the attribute information into a hierarchy. Information processing methods.

[0254] (6) A hierarchical classifying unit that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into a prediction point for deriving a difference value between the attribute information and a prediction value of the attribute information and a reference point used for deriving the prediction value, by recursively repeating the classification of the reference point, The hierarchical unit sets the reference points based on a distribution pattern of points. Information processing device. (7) The hierarchical unit sets the reference points based on table information that designates points close to a center of gravity of the points for each distribution pattern of the points. (6) An information processing device. (8) The hierarchical unit sets the reference points based on table information that designates a predetermined point for each distribution pattern of the points. An information processing device according to (6) or (7). (9) The apparatus further includes an encoding unit that encodes information related to the table information. (8) An information processing device. (10) In a method for classifying attribute information for each point of a point cloud that represents a three-dimensional object as a set of points, a prediction point for deriving a difference value between the attribute information and a prediction value of the attribute information and a reference point used for deriving the prediction value are recursively repeated for the reference points, and the reference points are set based on a distribution pattern of the points when classifying the attribute information into a hierarchy. Information processing methods.

[0255] (11) a hierarchical classifying unit that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into a prediction point that derives a difference value between the attribute information and a prediction value of the attribute information and a reference point that is used to derive the prediction value, by recursively repeating the classification of the reference point; an encoding unit that encodes information regarding the setting of the reference point by the hierarchical unit; An information processing device comprising: (12) The encoding unit encodes information regarding the setting of the reference point for all points. An information processing device according to (11). (13) The encoding unit encodes information regarding the setting of the reference point for a point of a part of a hierarchy. An information processing device according to (11). (14) The encoding unit further encodes information regarding setting of the reference point for a point that satisfies a predetermined condition. An information processing device according to (13). (15) Classifying attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into prediction points for deriving a difference value between the attribute information and a prediction value of the attribute information and reference points used to derive the prediction value, recursively repeating the classification for the reference points, thereby hierarchizing the attribute information; Encoding information regarding the setting of said reference points Information processing methods.

[0256] (16) A hierarchical classifying unit that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into a prediction point that derives a difference value between the attribute information and a prediction value of the attribute information and a reference point that is used to derive the prediction value, by recursively repeating the classification of the reference point, The hierarchical division unit alternately selects, for each layer, a point that is closer to a center of a bounding box and a point that is farther from the center of the bounding box from among the candidates for the reference point as the reference point. Information processing device. (17) The hierarchical unit selects, from among the candidates, a point that is closer to a center of the bounding box and a point that is farther from the center of the bounding box based on a search order according to a position within the bounding box. An information processing device according to (16). (18) The search order is the order of distance from the center of the bounding box. An information processing device according to (17). (19) The search order is set for each of the eight regions obtained by dividing the bounding box. An information processing device according to (17). (20) When classifying attribute information for each point of a point cloud that represents a three-dimensional object as a set of points, into a prediction point for deriving a difference value between the attribute information and a prediction value of the attribute information and a reference point used to derive the prediction value, the attribute information is hierarchically classified by recursively repeating the classification for the reference point, and a point that is closer to a center of a bounding box and a point that is farthest from the center of the bounding box are alternately selected for each hierarchy as the reference point. Information processing methods. [Explanation of symbols]

[0257] 100 encoding device, 101 position information encoding unit, 102 position information decoding unit, 103 point cloud generation unit, 104 attribute information encoding unit, 105 bit stream generation unit, 111 hierarchical processing unit, 112 quantization unit, 113 encoding unit, 121 reference point setting unit, 122 reference relationship setting unit, 123 inversion unit, 124 weight value derivation unit, 200 decoding device, 201 encoded data extraction unit, 202 position information decoding unit, 203 attribute information decoding unit, 204 point cloud generation unit, 211 decoding unit, 212 inverse quantization unit, 213 inverse hierarchical processing unit< / lifting> < / octree>

Claims

1. a hierarchical classifier that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into a prediction point for deriving a difference value between the attribute information and a prediction value of the attribute information, and a reference point used for deriving the prediction value, by recursively repeating the classification of the reference point; The hierarchical division unit sets the reference point based on the center of gravity of the points. Information processing device.

2. The hierarchical division unit sets a point that is closer to the center of gravity among the candidate points as the reference point. The information processing device according to claim 1 .

3. The hierarchical division unit sets the reference point based on a center of gravity of points located within a predetermined range. The information processing device according to claim 1 .

4. The hierarchical division unit sets the reference point based on a predetermined search order from among a plurality of candidates having substantially the same conditions with respect to the center of gravity. The information processing device according to claim 1 .

5. For attribute information for each point of a point cloud that represents a three-dimensional object as a set of points, classification is performed recursively for the reference points into prediction points for deriving a difference value between the attribute information and a prediction value of the attribute information, and reference points used to derive the prediction value, and the reference points are set based on the center of gravity of the points when classifying the attribute information into a hierarchy. Information processing methods.

6. a hierarchical classifier that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into a prediction point for deriving a difference value between the attribute information and a prediction value of the attribute information, and a reference point used for deriving the prediction value, by recursively repeating the classification of the reference point; The hierarchical unit sets the reference points based on a distribution pattern of points. Information processing device.

7. The hierarchical unit sets the reference points based on table information that designates points close to a center of gravity of the points for each distribution pattern of the points. The information processing device according to claim 6.

8. The hierarchical unit sets the reference points based on table information that designates a predetermined point for each distribution pattern of the points. The information processing device according to claim 6.

9. The apparatus further includes an encoding unit that encodes information related to the table information. The information processing device according to claim 8.

10. For attribute information for each point of a point cloud that represents a three-dimensional object as a set of points, classification is repeated recursively for the reference points into prediction points for deriving a difference value between the attribute information and a prediction value of the attribute information, and reference points used to derive the prediction value, and when classifying the attribute information into a hierarchy, the reference points are set based on the distribution state of the points. Information processing methods.

11. a hierarchical classifier that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into prediction points for deriving a difference value between the attribute information and a prediction value of the attribute information, and reference points used to derive the prediction value, recursively for the reference points, thereby classifying the attribute information into a hierarchy; an encoding unit that encodes information regarding the setting of the reference point by the hierarchical unit; An information processing device comprising:

12. The encoding unit encodes information regarding the setting of the reference point for all points. The information processing device according to claim 11.

13. The encoding unit encodes information regarding the setting of the reference point for points of a part of layers. The information processing device according to claim 11.

14. The encoding unit further encodes information regarding the setting of the reference point for a point that satisfies a predetermined condition. The information processing device according to claim 13.

15. classifying attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into prediction points for deriving a difference value between the attribute information and a prediction value of the attribute information, and reference points used to derive the prediction value, recursively for the reference points, thereby hierarchizing the attribute information; Encoding information regarding the setting of said reference points Information processing methods.

16. a hierarchical classifier that classifies attribute information for each point of a point cloud that represents a three-dimensional object as a set of points into a prediction point for deriving a difference value between the attribute information and a prediction value of the attribute information, and a reference point used for deriving the prediction value, by recursively repeating the classification of the reference point; The hierarchical division unit alternately selects, for each layer, a point that is closer to a center of a bounding box and a point that is farther from the center of the bounding box from among the candidates for the reference point as the reference point. Information processing device.

17. The hierarchical unit selects, from among the candidates, a point that is closer to a center of the bounding box and a point that is farther from the center of the bounding box based on a search order according to a position within the bounding box. The information processing device according to claim 16.

18. The search order is the order of distance from the center of the bounding box. The information processing device according to claim 17.

19. The search order is set for each of the eight regions obtained by dividing the bounding box. The information processing device according to claim 17.

20. For attribute information for each point of a point cloud that expresses a three-dimensional object as a collection of points, classification is repeated recursively for the reference points into prediction points for deriving a difference value between the attribute information and a prediction value of the attribute information, and reference points used to derive the prediction value, and when hierarchizing the attribute information, a point that is closer to the center of a bounding box and a point that is farthest from the center of the bounding box are alternately selected for each hierarchy as the reference point from among the reference point candidates. Information processing methods.

Images