Encoding device and method, and decoding device and method

Abstract

The present disclosure pertains to an information processing device and method that make it possible to more reliably decode attribute data. A neighbor point search for setting a reference point to be referenced when deriving a predicted value of attribute data of a node subject to processing is executed only with respect to, among nodes of a tree structure in which the nodes comprise attribute data of each point of a point cloud representing a three-dimensionally shaped object as a set of points, and in which slices that are mutually independently encodable node groups are formed, nodes that are decoded before the node subject to processing during decoding. The present disclosure can be applied to, for example, an information processing device, an encoding device, a decoding device, an electronic apparatus, an information processing method, or a program.

Description

[Technical Field]

[0001] The present disclosure provides: encoding Apparatus and method and composite device and method In particular, it has been made possible to decode attribute data more reliably. encoding Apparatus and method and composite device and method Regarding. [Background technology]

[0002] Conventionally, methods for encoding 3D data representing a three-dimensional structure such as a point cloud have been considered (see, for example, Non-Patent Document 1). Also, for the geometry data (position information) of this point cloud, methods have been considered for realizing parallel decoding or decoding some node groups by signaling a flag that divides the octree into node groups (see, for example, Non-Patent Document 2).

[0003] Furthermore, a method has been devised in which, for attribute data (attribute information) of a point cloud, a predicted value of a processing target point is derived using attribute data of neighboring points, and the difference between the attribute data of the processing target point and the predicted value is encoded. For example, a method called "Lifting" has been devised as such an encoding method (see, for example, Non-Patent Document 3). In such an encoding method, a neighboring point search is performed to set a point whose attribute data is to be referenced in order to derive a predicted value. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] R. Mekuria, Student Member IEEE, K. Blom, P. Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video",tcsvt_paper_submitted_february.pdf [Non-patent document 2] David Flynn, Khaled Mammou, "G-PCC: A hierarchical geometry slice structure", ISO / IEC JCTC1 / SC29 / WG11 MPEG / m54677, April 2020, Online [Non-patent document 3] Khaled Mammou, Alexis Tourapis, Jungsun Kim, Fabrice Robinet, Valery Valentin, Yeping Su, "Lifting Scheme for Lossy Attribute Encoding in TMC1", ISO / IEC JTC1 / SC29 / WG11 MPEG2018 / m42640, April 2018, San Diego, US Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method described in Non-Patent Document 3 does not limit the points to be searched. Therefore, for example, when node groups are formed in the tree structure of attribute data, there is a risk that nodes belonging to node groups that are not decoded may also be searched. This may cause the decoding of attribute data to fail.

[0006] The present disclosure has been made in light of such circumstances, and makes it possible to more reliably decode attribute data. [Means for solving the problem]

[0007] One aspect of this technology encoding device is a point cloud that represents a 3D object as a set of points. a layering unit that associates nodes of attribute data of the point cloud with a tree structure of geometry data of the point cloud; and a slice structure generation unit that divides the attribute data to generate a slice structure including a first slice and a second slice, Processing target during decryption point will be decrypted before point The target of the processing Points a neighboring point search unit that performs a neighboring point search to set a reference point to be referred to when deriving a predicted value of attribute data; point and the reference point set by the neighboring point search performed by the neighboring point search unit. of and a coding unit that codes a difference value between the predicted value and the calculated value. In the inter-reference of the node to be processed in the first slice, the neighboring point search unit excludes a node in the second slice that does not include an ancestor node of the node to be processed from the reference points. is.

[0008] One aspect of this technology Encoding method is a point cloud that represents a 3D object as a set of points. a node of attribute data of the point cloud is associated with a tree structure of geometry data of the point cloud, and the attribute data is divided to generate a slice structure including a first slice and a second slice; Processing target during decryption point will be decrypted before point The target of the processing Points A neighboring point search is performed to set a reference point to be referred to when deriving a predicted value of attribute data, and point and the reference point set by the neighbor point search. of and encoding the difference value between the predicted value and the calculated value. and in the neighboring point search, in an inter-reference of a node to be processed in the first slice, a node in the second slice that does not include an ancestor node of the node to be processed is excluded from the reference points. is.

[0009] Other aspects of the technology Decryption device is a point cloud that represents a 3D object as a set of points. Regarding Processing target Points decoding coded data in which a difference value between attribute data and a predicted value of the attribute data is coded; point a decoding unit for deriving the difference value of a layering unit that associates nodes of attribute data of the point cloud with a tree structure of geometry data of the point cloud; a slice structure generation unit that divides the attribute data to generate a slice structure including a first slice and a second slice; The decoding unit decodes the processed data. point was decrypted earlier than pointThe target of the processing point a neighboring point search unit that performs a neighboring point search to set a reference point to be referred to when deriving the predicted value of the difference value derived by the decoding unit, and a neighboring point search unit that performs a neighboring point search to set a reference point to be referred to when deriving the predicted value of the difference value derived by the decoding unit. of and the predicted value derived using point a restoration unit that restores the attribute data of the decoding target area restored by the restoration unit; point and a correlation unit that correlates the attribute data with the geometry data of the points. The neighboring point search unit excludes a node in the second slice that does not include an ancestor node of the node to be processed in the first slice from the inter-reference of the node to be processed. is.

[0010] Other aspects of the technology Decryption method is a point cloud that represents a 3D object as a set of points. Regarding Processing target Points decoding coded data in which a difference value between attribute data and a predicted value of the attribute data is coded; point deriving the difference value of Associating nodes of attribute data of the point cloud with a tree structure of geometry data of the point cloud, dividing the attribute data to generate a slice structure including a first slice and a second slice, Processing target point was decrypted earlier than point The target of the processing point A neighboring point search is performed to set a reference point to be referred to when deriving the predicted value of the difference value, and the derived difference value and the reference point set by the neighboring point search are compared. of and the predicted value derived using point The attribute data of the restored decoding target area is restored. point and associating the attribute data with the geometry data of the point. In the neighboring point search, in the inter-reference of the processing target node in the first slice, a node in the second slice that does not include an ancestor node of the processing target node is excluded from the reference points. is.

[0011] In an information processing device and method according to one aspect of the present technology, a point cloud that represents a three-dimensional object as a set of points is generated. a node of attribute data of the point cloud is associated with a tree structure of geometry data of the point cloud, and the attribute data is divided to generate a slice structure including a first slice and a second slice; Processing target during decryption point will be decrypted before point Targeting the processing target Points A nearby point search is performed to set the reference point to be used when deriving the predicted value of the attribute data, and the processing target point Attribute data and reference points set by the nearest neighbor search of The difference between the predicted value and the calculated value is encoded. In the neighbor point search, in the inter-reference of the node to be processed in the first slice, nodes in the second slice that do not include the ancestor node of the node to be processed are excluded from the reference points.

[0012] In another aspect of the present technology, an information processing device and a method thereof include: Regarding Processing target Points The coded data in which the difference between the attribute data and the predicted value of that attribute data is coded is decoded, and the processing target point The difference value of is derived, The nodes of the attribute data of the point cloud are associated with the tree structure of the geometry data of the point cloud, and the attribute data is divided to generate a slice structure including a first slice and a second slice. Processing target point was decrypted earlier than point Targeting the Processing target point A neighborhood point search is performed to set a reference point to be used when deriving the predicted value of , and the derived difference value and the reference point set by the neighborhood point search are of The predicted value derived using point The attribute data of the restored target area is point The attribute data of the point is associated with the geometry data of the point. In the neighbor point search, in the inter-reference of the node to be processed in the first slice, nodes in the second slice that do not include the ancestor node of the node to be processed are excluded from the reference points. [Brief explanation of the drawings]

[0013] [Figure 1] 1A and 1B are diagrams illustrating examples of a tree structure and a slice structure. [Figure 2] FIG. 10 is a diagram illustrating an example of a hierarchical method for attribute data. [Figure 3] FIG. 10 is a diagram illustrating an example of a slice structure of geometry data. [Figure 4] FIG. 10 is a diagram illustrating an example of a reference target of attribute data. [Figure 5]FIG. 10 is a diagram illustrating an example of a method for searching for neighboring points. [Figure 6] FIG. 10 is a diagram illustrating an example of a search target for inter-LoD reference. [Figure 7] FIG. 10 is a diagram illustrating an example of a search target for inter-LoD reference. [Figure 8] FIG. 10 is a diagram illustrating an example of a search target for intra-LoD reference. [Figure 9] FIG. 10 is a diagram illustrating an example of decoding some slices of geometry data. [Figure 10] FIG. 10 is a diagram illustrating an example of decoding some slices of attribute data. [Figure 11] FIG. 10 is a diagram illustrating an example of a method for associating data. [Figure 12] FIG. 10 is a diagram illustrating an example of a method for associating data. [Figure 13] FIG. 10 is a diagram illustrating an example of a method for associating data. [Figure 14] FIG. 1 is a block diagram illustrating an example of the main configuration of an encoding device. [Figure 15] 10 is a block diagram showing an example of the main configuration of an attribute data encoding unit. FIG. [Figure 16] 10 is a flowchart illustrating an example of the flow of an encoding process. [Figure 17] 10 is a flowchart illustrating an example of the flow of an attribute data encoding process. [Figure 18] FIG. 2 is a block diagram illustrating an example of the main configuration of a decoding device. [Figure 19] FIG. 10 is a block diagram showing an example of the main configuration of an attribute data decoding unit. [Figure 20] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 21] 10 is a flowchart illustrating an example of the flow of an attribute data decoding process. [Figure 22] 10 is a flowchart illustrating an example of the flow of a point cloud generation process. [Figure 23]10 is a flowchart illustrating an example of the flow of a point cloud generation process. [Figure 24] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Limitations of neighbor search 2. Mapping Restrictions 3. First embodiment (encoding device) 4. Second embodiment (decoding device) 5. Supplementary Notes

[0015] <1. Limitations on neighbor point search> <References supporting technical content and terminology> The scope of disclosure of the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that were publicly known at the time of filing.

[0016] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above)

[0017] In other words, the contents of the above-mentioned non-patent documents and the contents of other documents referenced in the above-mentioned non-patent documents are also used as the basis for determining the support requirements.

[0018] <Point Cloud> Conventional 3D data includes point clouds, which represent three-dimensional structures (objects with three-dimensional shapes) as a collection of numerous points. Point cloud data (also referred to as point cloud data) consists of position information (also referred to as geometry) and attribute information (also referred to as attributes) for each point. Attributes can include any information. For example, they can include color information, reflectance information, normal information, etc. for each point. Thus, point clouds have a relatively simple data structure, and by using a sufficient number of points, they can represent any three-dimensional structure with sufficient accuracy.

[0019] Point cloud data has a relatively large amount of data. Furthermore, the amount of data increases in proportion to the number of points. This increases the load when transmitting or storing point cloud data, for example. Therefore, voxelization was considered to reduce the amount of data. A voxel is a three-dimensional region for quantizing geometry (positional information).

[0020] That is, the three-dimensional area (also called a bounding box) containing the point cloud is divided into small three-dimensional areas called voxels, and each voxel indicates whether it contains a point. By doing this, the position of each point is quantized in voxel units. Therefore, it is possible to suppress an increase in the amount of information (typically, to reduce the amount of information).

[0021] The resolution of the geometry (i.e., the number of points) depends on the size of this voxel (in other words, the number of voxels). The smaller the voxel size (the more voxels), the higher the resolution of the geometry, as the number of points increases.

[0022] Taking advantage of this property, it was thought that voxels could be divided recursively to create a tree structure for the geometry. This method allows the resolution of the geometry to be variable. In other words, the geometry can be represented not only at the highest resolution corresponding to the lowest level of the tree structure, but also at lower resolutions corresponding to the intermediate levels of the tree structure.

[0023] Furthermore, in order to suppress the increase in load when transmitting or storing point cloud data, a method of encoding and decoding point cloud data has been devised. For example, when transmitting point cloud data, the transmitting side encodes the point cloud data to generate a bit stream, transmits the bit stream, and the receiving side decodes the bit stream to generate point cloud data. In this way, the amount of data during transmission can be reduced, and the increase in load can be suppressed. The same applies when storing point cloud data.

[0024] As mentioned above, a point cloud consists of geometry and attributes, which means that the data of the geometry and attributes are encoded and decoded separately.

[0025] By encoding geometry data (also referred to as geometry data) according to the above-described tree structure, the encoded geometry data can be decoded in a scalable manner in terms of resolution. In other words, resolution scalability can be achieved in the decoding process. For example, partial decoding is possible, such as decoding only nodes from the top layer of the tree structure up to an arbitrary layer (without decoding nodes in lower layers). In other words, geometry with a desired resolution can be obtained by decoding only the necessary parts without decoding all of the encoded geometry data.

[0026] <Node grouping> Incidentally, Non-Patent Document 2 discloses a method for signaling a flag that divides the tree structure (octree) of this point cloud into node groups. This method enables independent decoding for each node group. Therefore, for example, the decoding process can be divided into node groups and parallelized. It is also possible to decode only some node groups. For example, by dividing node groups into subregions of a 3D region, it is possible to decode only the geometry of a certain region. In other words, scalability for regions can be achieved in the decoding process. In other words, the geometry of a desired region can be obtained by decoding only the necessary part without decoding all the encoded data of the geometry.

[0027] This node group is also called a slice. In other words, a slice is a group of nodes in the tree structure of geometry. Geometry data can be coded independently for each slice. In other words, coded data of geometry data can be decoded independently for each slice. The tree structure of geometry may be any tree structure. For example, it may be an Octree or KD Tree, or may be something else.

[0028] Another example of a slice structure set in geometry data is shown in Figure 1. In Figure 1, thin solid lines indicate a tree structure, and the branching points of these solid lines indicate nodes. A group of nodes surrounded by thick lines forms one slice. LoD indicates the hierarchy of the tree structure.

[0029] That is, all nodes of LoD0 to LoD4 belong to slice 1. Nodes in region 1 (gray area) of LoD5 to LoD7 belong to slice 2. Nodes in region 2 (white area) of LoD5 to LoD7 belong to slice 3. Nodes in region 3 (gray area) of LoD5 to LoD7 belong to slice 4. Nodes in region 4 (white area) of LoD5 to LoD7 belong to slice 5.

[0030] The dotted arrows indicate the order of encoding and decoding. For example, each node of LoD0 to LoD4 is coded for each layer in order from the highest layer to the lowest layer for the entire region. Each node of LoD5 to LoD7 is coded for each layer in order from the highest layer to the lowest layer for each region of Region 1 to Region 4. The same applies to the decoding order.

[0031] When such a slice structure is formed, it is possible to decode only some of the slices. For example, if geometry data of a point in region 3 at LoD7 resolution is required, it is sufficient to decode the nodes in slices 1 and 4. In other words, in this case, it is not necessary to decode the nodes in slices 2, 3, and 5. This reduces the amount of decoding processing compared to decoding all nodes in LoD7.

[0032] <Lifting> In response to this, methods have been devised for encoding and decoding attributes by assuming that the geometry, including degradation due to encoding, is known and utilizing the positional relationships between points. RAHT (Region Adaptive Hierarchical Transform) and a method using a transformation called Lifting, as described in Non-Patent Document 2, have been considered as methods for encoding such attributes. By applying these technologies, attributes can also be hierarchically organized, like a geometry octree.

[0033] For example, in the case of lifting, the attribute data of each point is encoded as a difference value from a predicted value derived by referring to the attribute data of other points. Let's explain this in two dimensions, for example. In Figure 2A, each circle represents a point. Let's assume that the points have the following positional relationship. The attribute data of each point is encoded and decoded as a difference value from a predicted value derived by referring to the attribute data of other points, as shown by the arrows in Figure 2B.

[0034] As with geometry data, scalable decoding of attribute data can be achieved by hierarchizing (tree-structuring) the attribute data of each point. Furthermore, scalable decoding of point cloud data can be achieved by matching this attribute tree structure to the geometry tree structure. In this case, the attribute data of each point can be hierarchized using geometry.

[0035] For example, as shown in FIG. 2C, a predicted point is selected so that attribute information remains in a voxel in the layer one level higher (LoD) than the voxel containing the point. In FIG. 2C, squares represent voxels and circles represent points. Voxel 20 is divided into four to form voxels in the layer one level lower, and the voxel to the lower right of voxel 20 is divided into four to form voxels in an even lower layer. One of the four points (three white points and one gray point) in the voxel in the lowest layer is left in a voxel in the layer one level higher (the gray point). In other words, one black point and three gray points remain in the voxel obtained by dividing voxel 20 into four. Similarly, one of these points (the black point) is left in a voxel in the layer one level higher (i.e., voxel 20). By hierarchizing attribute data in this way, a hierarchical structure (tree structure) can be formed for attribute data as well, similar to that for geometry data.

[0036] <Neighborhood search> 2B, a search for neighboring points (also called a neighboring point search) is performed to derive a predicted value for the attribute of a target point by referencing the attribute data of other points (neighboring points). There are two ways of referencing to derive the predicted value: inter-referencing (also called inter-LoD referencing), which refers to points that belong to a LoD different from the LoD to which the target point belongs, and intra-referencing (also called intra-LoD referencing), which refers to points that belong to the same LoD as the target point.

[0037] <Attribute node grouping> As mentioned above, geometry data can group nodes in a tree structure (divide them into slices). Figure 3 shows an example of such a slice structure. In the example of Figure 3, the entire area of ​​the upper 2 LoD is sliced ​​into one slice (slice #1). In contrast, the lower 2 LoD is divided into two areas, and each area is sliced ​​(slice #2, slice #3). For example, nodes A to D in the bottom layer belong to slice #2, and nodes E to G belong to slice #3.

[0038] Similar to this slice division, it is possible to group nodes (slice division) in attribute data as well. This is expected to enable parallel encoding and decoding of attribute data, or to decode only some slices. Figure 4 shows an example of the slice structure of attribute data in this case. Nodes A to G in Figure 4 correspond to those in Figure 3. In the example of Figure 4, the entire region of the upper 2 LoD is sliced ​​into one (slice #4), just like the geometry data in Figure 3. In contrast, the lower 2 LoD is divided into two regions, and each region is sliced ​​(slice #5, slice #6).

[0039] However, the attribute encoding and decoding methods described in Non-Patent Document 2 do not support such slice structures.

[0040] For example, since the attribute data of each node is decoded for each LoD and the target of the neighbor point search is not limited, the attribute data of points in the entire area are subject to the neighbor point search. Therefore, there is a risk that the attribute data of points in areas that are not subject to decoding (i.e., nodes belonging to slices that are not decoded) may be referenced.

[0041] For example, in the example of Fig. 4, when node F is the decoding target, in the method described in Non-Patent Document 2, the target of neighboring point search in inter-reference is the nodes within the dashed-dotted line frame. That is, among nodes A to G, nodes A, D, E, and G are search targets. Also, the target of neighboring point search in intra-reference is the nodes within the dashed-dotted line frame. That is, among nodes A to G, nodes B and C are search targets.

[0042] When the image is divided into slices as shown in FIG. 4, it is sufficient to decode at least slices #4 and #6 to decode node F. In other words, slice #5 does not need to be decoded. However, in the case of the method described in Non-Patent Document 2, as mentioned above, there is a risk that the node of slice #5 may also be referenced. In that case, it may become impossible to decode each slice independently, and it may become impossible to decode attribute data.

[0043] Therefore, as shown in the top row of the table in Fig. 5, in the tree structure (also called attribute tree) of attribute data to which slice division has been applied, a neighborhood search is performed only on decoded nodes (Method 1). The decoded nodes are nodes that are decoded before the node to be processed during decoding. Furthermore, "neighborhood search is performed only on decoded nodes" means that among the nodes in the tree structure, "decoded nodes" are the targets of neighborhood search, and other nodes (nodes that have not been decoded) are not the targets of neighborhood search.

[0044] For example, in an information processing method, the attribute data of each point in a point cloud that represents a three-dimensional object as a collection of points is treated as a node, and among the nodes in a tree structure that form slices, which are node groups that can be coded independently of each other, only nodes that are decoded before the node to be processed during decoding are targeted, and a neighboring point search is performed to set a reference point to be referenced when deriving a predicted value of the attribute data of the node to be processed, and the difference value between the attribute data of the node to be processed and the predicted value derived using the attribute data of the reference point set by the neighboring point search is coded.

[0045] For example, an information processing device may include a neighboring point search unit that performs a neighboring point search to set a reference point to be referenced when deriving a predicted value of the attribute data of a target node, targeting only nodes that are decoded before the target node during decoding among nodes in a tree structure in which attribute data of each point in a point cloud that represents a three-dimensional object as a collection of points is defined as a node, and an encoding unit that encodes the difference value between the attribute data of the target node and the predicted value derived using the attribute data of the reference point set by the neighboring point search unit.

[0046] For example, in an information processing method, the attribute data of each point in a point cloud that represents a three-dimensional object as a collection of points is treated as a node, and coded data in which the difference between the attribute data and the predicted value of that attribute data of a target node in a tree structure in which slices, which are node groups that can be coded independently of each other, is coded is decoded, the difference value of the target node is derived, and a neighboring point search is performed for only those nodes in the tree structure that were decoded before the target node in order to set a reference point to be used when deriving the predicted value of the target node, and the attribute data of the target node is restored by adding the derived difference value and the predicted value derived using the attribute data of the reference point set by the neighboring point search.

[0047] For example, an information processing device may include a decoding unit that decodes coded data in which the attribute data of each point in a point cloud, which represents a three-dimensional object as a collection of points, is a node, and the differential value between the attribute data and the predicted value of the attribute data of a target node in a tree structure in which slices, which are node groups that can be coded independently of each other, is coded, and derives the differential value of the target node; a neighboring point search unit that performs a neighboring point search for only nodes in the tree structure that were decoded by the decoding unit before the target node in order to set a reference point to be used when deriving the predicted value of the target node; and a restoration unit that restores the attribute data of the target node by adding the differential value derived by the decoding unit to the predicted value derived using the attribute data of the reference point set by the neighboring point search unit.

[0048] Note that "performing a neighbor search only on nodes that are decoded (or have been decoded) before the node to be processed" means that a neighbor search is performed on "nodes in the tree structure that are decoded (or have been decoded) before the node to be processed," excluding other nodes. By limiting the targets of the neighbor search as described above, attribute data can be decoded independently for each slice, and attribute data can be decoded more reliably.

[0049] The tree structure of the attribute data may be such that the attribute data of each point hierarchically organized based on the geometry data is used as a node.

[0050] For example, the tree structure of the attribute data may be structured based on the geometry data, with the attribute data of each point being a node, hierarchically arranged so that the voxel in which the point in the processing target layer exists also contains a point in the voxel one layer above it.

[0051] Alternatively, the geometry data may form a hierarchical tree structure based on the resolution of the geometry, and each node in the tree structure of the attribute data may correspond to each node in the tree structure of the geometry data. Furthermore, slices, which are node groups that can be coded independently of each other, may be formed in the tree structure of the geometry data, and the structure of the slices formed in the tree structure of the attribute data may correspond to the structure of the slices formed in the tree structure of the geometry data.

[0052] By matching the tree structure of the attribute data to the tree structure of the geometry data in this way, scalable decoding of the point cloud data can be realized. Also, by matching the slice structure of the attribute data to the slice structure of the geometry data, the point cloud data can be restored independently for each slice.

[0053] <Inter-reference> As shown in the second row from the top of the table in Fig. 5, a neighboring point search for inter-LoD reference may be performed on decoded nodes (method 1-1). For example, in an information processing device, a neighboring point search unit may perform a neighboring point search on only nodes in a higher hierarchy than the target node among the nodes in the tree structure of attribute data that are decoded before the target node during decoding. Alternatively, in an information processing device, a neighboring point search unit may perform a neighboring point search on only nodes in a higher hierarchy than the target node among the nodes in the tree structure of attribute data that are decoded by a decoding unit before the target node.

[0054] In this way, attribute data to which inter-referencing is applied during decoding can be decoded independently for each slice, and the attribute data can be decoded more reliably.

[0055] <Limiting neighboring point search targets for inter-reference based on slice structure> As a neighboring point search for the inter-reference, a neighboring point search may be performed on nodes belonging to a decoded slice (Method 1-1-1), as shown in the third row from the top of the table in Fig. 5. This decoded slice refers to a slice (a node of a slice) that is decoded before the node to be processed during decoding.

[0056] For example, in an information processing device, a neighboring point search unit may perform a neighboring point search only on nodes in the tree structure of attribute data that belong to the same slice as the target node, are decoded earlier than the target node during decoding, and are in a higher hierarchical level than the target node during decoding, and on nodes belonging to a slice that is decoded earlier than the slice to which the target node belongs during decoding. Alternatively, in an information processing device, a neighboring point search unit may perform a neighboring point search only on nodes in the tree structure of attribute data that belong to the same slice as the target node, are decoded earlier than the target node during decoding by the decoding unit, and are in a higher hierarchical level than the target node during decoding, and on nodes belonging to a slice that is decoded earlier than the slice to which the target node belongs during decoding.

[0057] In this case, for example, if node F in the tree structure of Fig. 4 is to be decoded, a neighboring point search for inter-reference is performed on the nodes within the dashed-dotted line frame shown in Fig. 6. That is, among nodes A to G, nodes A, E, and G are targeted for the search. In other words, nodes B to D, i.e., nodes belonging to slice #5, are excluded from the neighboring point search (they are not set as nodes that reference attribute data).

[0058] Therefore, in this case, the attribute data can be decoded independently for each slice, and the attribute data can be decoded more reliably.

[0059] <Restricting neighbor search targets for inter-reference based on tree structure> Furthermore, as a neighbor point search for inter-referencing, a neighbor point search may be performed on ancestor nodes in the tree structure of attribute data, as shown in the fourth row from the top of the table in FIG. 5 (Method 1-1-2). Here, an ancestor node is a node in a higher LoD to which the target node belongs directly or indirectly in the tree structure of attribute data. For example, in the tree structure shown in FIG. 7, the lines between nodes indicate the parent-child relationship between the nodes. A node connected to the target node (reference node) by a line in the direction from the node to a higher LoD, directly or indirectly, is called an ancestor node of the reference node.

[0060] A directly connected node is a node that is connected to a certain node by a line without going through any other nodes. For example, node E is a node that is directly connected to node F. An indirectly connected node is a node that is connected to a certain node by a line through another node. For example, node G is a node that is indirectly connected to node F (connected through node E, etc.). In other words, nodes E and G are ancestor nodes of node F.

[0061] Although nodes A to D can also be said to be indirectly connected to node F (connected via node G, etc.), these nodes belong to a lower LoD than node G. In other words, they are not (indirectly) connected to node F in the direction toward a higher LoD, and therefore are not ancestor nodes of node F.

[0062] For example, in the information processing device, the neighboring point search unit may perform a neighboring point search only on ancestor nodes of the processing target node among the nodes in the tree structure of the attribute data.

[0063] That is, in this case, for example, if node F in the tree structure of FIG. 4 is to be decoded, a neighboring point search for inter-reference is performed on the nodes within the dashed-dotted line frame shown in FIG. 7. That is, among nodes A to G, nodes E and G are searched. In other words, nodes A to D are excluded from the neighboring point search (they are not set as nodes that reference attribute data). That is, in this case, nodes belonging to slice #5 are also excluded from the neighboring point search. Note that, for example, if node C belonging to slice #5 is to be decoded, a neighboring point search for inter-reference is performed on the nodes within the dashed-dotted line frame shown in FIG. 7. That is, among nodes A to G, nodes A, D, and G are searched. In other words, nodes B, E, and F are excluded from the neighboring point search (they are not set as nodes that reference attribute data). That is, in this case, nodes belonging to slice #6 are excluded from the neighboring point search.

[0064] Therefore, in this case, the attribute data can be decoded independently for each slice, and the attribute data can be decoded more reliably.

[0065] <See Intranet> As shown in the fifth row from the top of the table in Fig. 5, a neighboring point search for intra-LoD reference (intra reference) may be performed on decoded nodes (Method 1-2). For example, in an information processing device, a neighboring point search unit may perform a neighboring point search on only nodes that are decoded earlier than the target node during decoding, among nodes at the same hierarchical level as the target node in the tree structure of attribute data. Alternatively, in an information processing device, a neighboring point search unit may perform a neighboring point search on only nodes that are decoded earlier than the target node by a decoding unit, among nodes at the same hierarchical level as the target node in the tree structure of attribute data.

[0066] In this way, attribute data to which intra-reference is applied during decoding can be decoded independently for each slice, and the attribute data can be decoded more reliably.

[0067] <Limiting neighboring point search targets for intra-frame referencing based on slice structure> As a neighboring point search for the intra-reference, a neighboring point search may be performed on decoded nodes in the current LoD of the current slice, as shown in the bottom row of the table in Fig. 5 (Method 1-2-1). The current slice is the slice to which the current node belongs. The current LoD is the LoD to which the current node belongs. The decoded node is a node that is decoded before the current node during decoding.

[0068] For example, in an information processing device, a neighboring point search unit may perform a neighboring point search only on nodes in the tree structure of attribute data that belong to the same slice as the target node, are decoded earlier than the target node during decoding, and are at the same hierarchical level as the target node. Alternatively, in an information processing device, a neighboring point search unit may perform a neighboring point search only on nodes in the tree structure of attribute data that belong to the same slice as the target node, are decoded earlier than the target node by the decoding unit, and are at the same hierarchical level as the target node.

[0069] In this case, for example, if node F in the tree structure of Fig. 4 is to be decoded, a neighboring point search for intra-reference is performed on the nodes within the two-dot chain line frame shown in Fig. 8. That is, in this case, the node of the lowest LoD in slice #6 is the search target. In other words, nodes B and C, i.e., nodes belonging to slice #5, are excluded from the neighboring point search (they are not set as nodes that reference attribute data).

[0070] Therefore, in this case, the attribute data can be decoded independently for each slice, and the attribute data can be decoded more reliably.

[0071] <2. Mapping Restrictions> <Matching geometry and attributes> By the way, when the geometry data and attribute data are decoded, the geometry data and attribute data are associated with each other. That is, the geometry and attributes are combined (linked) for each point, and point cloud data is generated.

[0072] However, when the attribute data is decoded by dividing it into slices as described above, there is a risk that the data will not correspond to the geometry.

[0073] For example, assume that slices #1 and #3 of the geometry data are decoded, but slice #2 is not, as shown in Figure 9. In this case, the number of decoded geometry data points is 1, 2, 2, and 3 from the higher LoD to the lower LoD, as shown in Figure 9.

[0074] In contrast, as shown in FIG. 10, suppose slices #4 and #6 of the attribute data are decoded, but slice #5 is not. In this case, the number of decoded attribute data is 1 color, 2 colors, 3 colors, and 4 colors from the higher LoD to the lower LoD, as shown in FIG. 10. In other words, in the lower two LoDs, the number of decoded geometry data does not match the number of attribute data. Therefore, for the lower two LoDs, it is not possible to associate geometry data with attribute data, and there is a risk that point cloud data cannot be generated.

[0075] <Matching restrictions based on slice structure> Therefore, as shown in the top row of the table in Fig. 11, in associating geometry data with attribute data (i.e., generating point cloud data), the attributes of the decoding target area are associated with the geometry (Method 2). The decoding target area is the area corresponding to the slice to be decoded (the area where the points corresponding to the nodes belonging to that slice are located).

[0076] For example, in an information processing method, attribute data of each point of a point cloud that represents a three-dimensional object as a collection of points is treated as a node, and coded data in which the difference between attribute data and a predicted value of that attribute data is coded for a node to be processed in a tree structure in which slices, which are node groups that can be coded independently of each other, are formed is decoded, the difference value of the node to be processed is derived, and a neighboring point search is performed for only those nodes in the tree structure that were decoded before the node to be processed to set a reference point to be used when deriving the predicted value of the node to be processed, and the attribute data of the node to be processed is restored by adding the derived difference value and the predicted value derived using the attribute data of the reference point set by the neighboring point search, and the restored attribute data of the point in the decoding target area is associated with the geometry data of that point.

[0077] For example, an information processing device may include a decoding unit that decodes encoded data in which the attribute data of each point in a point cloud, which represents a three-dimensional object as a collection of points, is a node, and the differential value between the attribute data and the predicted value of the attribute data of a processing target node in a tree structure in which slices, which are node groups that can be coded independently of each other, are coded, and derives the differential value of the processing target node; a neighboring point search unit that performs a neighboring point search for setting a reference point to be referenced when deriving the predicted value of the processing target node, targeting only nodes in the tree structure that were decoded by the decoding unit before the processing target node; a restoration unit that restores the attribute data of the processing target node by adding the differential value derived by the decoding unit to the predicted value derived using the attribute data of the reference point set by the neighboring point search by the neighboring point search unit; and a correspondence unit that associates the attribute data of the point in the decoding target area restored by the restoration unit with the geometry data of the point.

[0078] By doing so, it is possible to more reliably associate the geometry data with the attribute data, and it is possible to more reliably generate point cloud data.

[0079] <Remove after matching> As shown in the second row from the top of the table in Fig. 11, the geometry data of the non-decoding region may also be included in the correspondence, and then points in the non-decoding region may be removed based on the geometry (Method 2-1). Here, the non-decoding region refers to a region in which points corresponding to nodes belonging to slices that are not the decoding region are located. In other words, the non-decoding region refers to a region in which points corresponding to nodes belonging to slices that are the decoding region are not located. Note that the region in which points corresponding to nodes belonging to slices that are the decoding region are located is referred to as the decoding region.

[0080] For example, as shown in the third row from the top of the table in Figure 11, after matching the geometry data with the attribute data, i.e., after generating the point cloud data, the decoded geometry may be used to remove points in areas not to be decoded (Method 2-1-1).

[0081] For example, in an information processing device, the matching unit may match the attribute data and geometry data of all points restored by the restoration unit, and remove points in areas not to be decoded from among the points for which the attribute data and geometry data are matched.

[0082] In this case, for example, the attribute data obtained by decoded slices #4 and #6 as shown in Fig. 10 is associated with geometry data including node A as shown in Fig. 12. By doing so, the number of geometry data and attribute data matches even in the lower 2LoD as shown in Fig. 12, making it possible to associate them. After the association, node A can be removed by removing points in the non-decoding target area based on the geometry data.

[0083] In other words, by doing this, only the points in the decoding target area remain, which makes it possible to more reliably associate geometry data with attribute data and more reliably generate point cloud data.

[0084] <Remove before mapping> As shown in the fourth row from the top of the table in Fig. 11, the attribute data of the region not to be decoded may be removed before associating the geometry data with the attribute data (Method 2-2). For example, as shown in the bottom row of the table in Fig. 11, the attribute data of the region not to be decoded may be removed by using the geometry associated with the node during LoD Generation, before associating the geometry data with the attribute data (Method 2-2-1).

[0085] For example, in an information processing device, the association unit may remove attribute data of points in the non-decoding target area restored by the restoration unit, and associate the attribute data of the points in the decoding target area with geometry data.

[0086] In this case, for example, when constructing a tree structure of attribute data, geometry is associated with each node of the attribute data. Intermediate resolution geometry data (i.e., geometry data of the same LoD) is associated with each node of the lowest LoD. Then, before associating the attribute data with the geometry data (i.e., generating point cloud data), the geometry of each node is used to remove attribute data of points in areas not to be decoded, as shown in FIG. 13. In the example of FIG. 13, the geometry of node A is located in the area to be decoded, so node A is removed. As a result, the lowest layer has three colors, and the number of geometry data and attribute data matches.

[0087] Therefore, the geometry data and the attribute data can be associated with each other. In other words, by doing so, the geometry data and the attribute data can be associated with each other more reliably, and point cloud data can be generated more reliably.

[0088] 3. First Embodiment <Encoding device> Fig. 14 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology is applied. The encoding device 100 shown in Fig. 14 is a device that encodes a point cloud (3D data). The present technology (for example, various methods described with reference to Figs. 1 to 13) can be applied to the encoding device 100.

[0089] Note that Fig. 14 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in encoding device 100, there may be processing units that are not shown as blocks in Fig. 14, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 14.

[0090] As shown in FIG. 14, the encoding device 100 includes a geometry data encoding unit 101, a geometry data decoding unit 102, a point cloud generating unit 103, an attribute data encoding unit 104, and a bitstream generating unit 105.

[0091] The geometry data encoding unit 101 encodes position information of the point cloud (3D data) input to the encoding device 100, and generates encoded data of the geometry data. This encoding method is arbitrary. For example, processing such as filtering and quantization for noise suppression (denoising) may be performed. The geometry data encoding unit 101 supplies the generated encoded data to the geometry data decoding unit 102 and the bitstream generation unit 105.

[0092] The geometry data decoding unit 102 acquires the coded data supplied from the geometry data encoding unit 101. The geometry data decoding unit 102 decodes the coded data to generate geometry data. Any decoding method may be used as long as it is compatible with the coding performed by the geometry data encoding unit 101. For example, processing such as filtering or inverse quantization for denoising may be performed. The geometry data decoding unit 102 supplies the generated geometry data (decoded result) to the point cloud generation unit 103.

[0093] The point cloud generation unit 103 acquires attribute data of the point cloud input to the encoding device 100 and geometry data (decoded result) supplied from the geometry data decoding unit 102. The point cloud generation unit 103 performs processing (recolor processing) to match the attribute data with the geometry data (decoded result). The point cloud generation unit 103 supplies the attribute data associated with the geometry data (decoded result) to the attribute data encoding unit 104.

[0094] The attribute data encoding unit 104 acquires the point cloud data (geometry data (decoded result) and attribute data) supplied from the point cloud generation unit 103. The attribute data encoding unit 104 uses the geometry data (decoded result) to encode the attribute data and generate coded data of the attribute data. The attribute data encoding unit 104 supplies the generated coded data to the bit stream generation unit 105.

[0095] The bitstream generation unit 105 acquires coded data of geometry data supplied from the geometry data encoding unit 101. The bitstream generation unit 105 also acquires coded data of attribute data supplied from the attribute data encoding unit 104. The bitstream generation unit 105 multiplexes these coded data to generate a bitstream including these coded data. The bitstream generation unit 105 outputs the generated bitstream to the outside of the encoding device 100. This bitstream is supplied to a decoding side device (for example, a decoding device described later) via, for example, any communication medium or any storage medium.

[0096] In such an encoding device 100, the present technology described above in <1. Limitations on Neighboring Point Search> may be applied to the attribute data encoding unit 104. That is, in this case, the attribute data encoding unit 104 encodes attribute data by a method to which the present technology described above in <1. Limitations on Neighboring Point Search> is applied.

[0097] With this configuration, the encoding device 100 can encode the attribute data so that it can be decoded independently for each slice, thereby more reliably decoding the attribute data.

[0098] These processing units (the geometry data encoding unit 101 to the bitstream generation unit 105) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Furthermore, each processing unit may have, for example, a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-described processing using a logic circuit and the other part by executing a program. The configurations of the processing units may be independent of each other. For example, some processing units may realize part of the above-described processing using a logic circuit, other processing units may execute a program to realize the above-described processing, and still other processing units may realize the above-described processing using both a logic circuit and by executing a program.

[0099] <Attribute data encoding part> Fig. 15 is a block diagram showing an example of the main configuration of the attribute data encoding unit 104 (Fig. 14). Note that Fig. 15 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, the attribute data encoding unit 104 may have processing units that are not shown as blocks in Fig. 15, or may have processing or data flows that are not shown as arrows, etc. in Fig. 15.

[0100] As shown in FIG. 15, the attribute data encoding unit 104 includes a layering unit 131 , a slice structure generating unit 132 , a neighboring point searching unit 133 , a prediction processing unit 134 , a quantizing unit 135 , and an encoding unit 136 .

[0101] The layering unit 131 executes processing related to layering of attribute data (LoD Generation). For example, the layering unit 131 acquires attribute data and geometry data (decoding results) supplied from the point cloud generation unit 103. The layering unit 131 layers the attribute data using the geometry data. For example, the layering unit 131 layers the attribute data so as to generate a tree structure similar to that of the geometry data. The layering unit 131 supplies the layered attribute data together with the geometry data to the slice structure generation unit 132.

[0102] The slice structure generation unit 132 executes processing related to the generation of a slice structure. For example, the slice structure generation unit 132 acquires attribute data and geometry data supplied from the layering unit 131. The slice structure generation unit 132 performs slice division on the attribute data to generate a slice structure. That is, the slice structure generation unit 132 groups nodes in the tree structure of the attribute data to form node groups. At this time, the slice structure generation unit 132 performs slice division so as to divide each node into regions using geometry data (that is, based on the geometry of the points). For example, the slice structure generation unit 132 generates a slice structure for the attribute data so as to have the same slice structure as that of the geometry data. The slice structure generation unit 132 supplies the attribute data for which the slice structure has been generated to the neighbor point search unit 133 together with the geometry data.

[0103] The neighboring point search unit 133 executes a process for searching for neighboring points by referencing attribute data in order to derive a predicted value of the processing target point. For example, the neighboring point search unit 133 acquires attribute data and geometry data supplied from the slice structure generation unit 132. The neighboring point search unit 133 also executes a neighboring point search based on the geometry data.

[0104] The neighboring point search unit 133 performs the neighboring point search by applying the present technology described above in <1. Restrictions on the neighboring point search>. For example, as described above in <Node grouping of attributes>, the neighboring point search unit 133 may perform the neighboring point search for setting a reference point to be referred to when deriving a predicted value of the attribute data of the node to be processed, targeting only nodes that are decoded before the node to be processed during decoding, among nodes in a tree structure in which slices, which are node groups that can be coded independently of each other, are formed.

[0105] The tree structure of the attribute data may have nodes that represent attribute data of each point hierarchically organized based on the geometry data. For example, the tree structure of the attribute data may have nodes that represent attribute data of each point hierarchically organized based on the geometry data so that a point also exists in a voxel one level higher than the voxel in which a voxel in the processing target level exists. Alternatively, the geometry data may form a hierarchical tree structure based on the resolution of the geometry, and each node in the tree structure of the attribute data may correspond to each node in the tree structure of the geometry data. Furthermore, slices, which are node groups that can be coded independently of each other, may be formed in the tree structure of the geometry data, and the structure of the slices formed in the tree structure of the attribute data may correspond to the structure of the slices formed in the tree structure of the geometry data.

[0106] Furthermore, as described above in <Inter Reference>, the neighboring point search unit 133 may perform a neighboring point search only on nodes that are higher in the hierarchy than the node to be processed in the tree structure of the attribute data and that are decoded before the node to be processed during decoding.

[0107] As a neighboring point search for the inter-referencing, the neighboring point search unit 133 may perform a neighboring point search only on nodes in the tree structure of the attribute data that belong to the same slice as the target node to be processed, are decoded before the target node during decoding, and are in a higher hierarchical level than the target node, as well as nodes that belong to a slice that is decoded before the slice to which the target node belongs during decoding, as described above in <Limiting neighboring point search targets for inter-referencing based on slice structure>.

[0108] Furthermore, as a neighbor point search for the inter-reference, the neighbor point search unit 133 may perform a neighbor point search by targeting only ancestor nodes of the node being processed among the nodes in the tree structure of the attribute data, as described above in <Limiting neighbor point search targets for inter-reference based on tree structure>.

[0109] Furthermore, as described above in <Intra Reference>, the neighboring point search unit 133 may perform a neighboring point search only for nodes at the same level as the node to be processed in the tree structure of the attribute data, which nodes will be decoded before the node to be processed during decoding.

[0110] As a neighboring point search for the inter-reference, the neighboring point search unit 133 may perform a neighboring point search only on nodes in the tree structure of the attribute data that belong to the same slice as the node to be processed, that are decoded before the node to be processed during decoding, and that are at the same hierarchical level as the node to be processed, as described above in <Limiting neighboring point search targets for intra-reference based on slice structure>.

[0111] The neighboring point search unit 133 supplies the search results to the prediction processing unit 134 together with the attribute data and geometry data.

[0112] The prediction processing unit 134 executes processing related to the prediction of attribute data. For example, the prediction processing unit 134 acquires the search results of the neighboring point search, attribute data, and geometry data supplied from the neighboring point search unit 133. The prediction processing unit 134 uses this information to derive a predicted value of the attribute data of the node to be processed. For example, the prediction processing unit 134 sets the searched neighboring points as a parent node or a grandparent node, and derives a predicted value of the attribute data of the node to be processed using the attribute data of these nodes.

[0113] Furthermore, the prediction processing unit 134 derives a difference value between the attribute data of the processing target node and the derived predicted value. Then, the prediction processing unit 134 supplies the derived difference value to the quantization unit 135.

[0114] The quantization unit 135 acquires the difference value supplied from the prediction processing unit 134. The quantization unit 135 quantizes the difference value. The quantization unit 135 supplies the quantized difference value to the encoding unit 136.

[0115] The encoding unit 136 acquires the quantized difference value supplied from the quantization unit 135. The encoding unit 136 encodes the quantized difference value to generate encoded data of the attribute data (difference value). This encoding method is arbitrary. The encoding unit 136 supplies the generated encoded data to the bitstream generation unit 105 (FIG. 14).

[0116] As described above, the neighboring point search unit 133 performs a neighboring point search by applying the present technology, and the encoding device 100 (the attribute data encoding unit 104) can encode the attribute data so that it can be decoded independently for each slice. Therefore, the attribute data can be decoded more reliably.

[0117] These processing units (layering unit 131 to encoding unit 136) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Furthermore, each processing unit may have, for example, a CPU, ROM, RAM, etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-described processing using a logic circuit and the other part by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may realize part of the above-described processing using a logic circuit, other processing units may execute a program to realize the above-described processing, and still other processing units may realize the above-described processing using both a logic circuit and by executing a program.

[0118] <Encoding process flow> Next, a description will be given of the processing executed by the encoding device 100. The encoding device 100 encodes point cloud data by executing an encoding process. An example of the flow of this encoding process will be described with reference to the flowchart in FIG.

[0119] When the encoding process starts, in step S101, the geometry data encoding unit 101 of the encoding device 100 encodes the geometry data of the input point cloud to generate encoded data of the geometry data.

[0120] In step S102, the geometry data decoding unit 102 decodes the coded data generated in step S101 to generate geometry data.

[0121] In step S103, the point cloud generation unit 103 performs recolor processing using the attribute data of the input point cloud and the geometry data (decoded result) generated in step S102, and associates the attribute data with the geometry data.

[0122] In step S104, the attribute data encoding unit 104 performs attribute data encoding processing to encode the attribute data that has been recolored in step S103, and generate encoded data of the attribute data.

[0123] In step S105, the bitstream generating unit 105 generates and outputs a bitstream including the coded data of the geometry data generated in step S101 and the coded data of the attribute data generated in step S104.

[0124] When the process of step S105 is completed, the encoding process ends.

[0125] The attribute data encoding process executed in step S104 of such encoding process can be applied with the present technology described above in <1. Limitation of Neighboring Point Search>. That is, in this case, the attribute data encoding unit 104 executes the attribute data encoding process by applying the present technology described above in <1. Limitation of Neighboring Point Search>, and encodes the attribute data.

[0126] By performing the processing at each step in this manner, the encoding device 100 can encode the attribute data so that it can be decoded independently for each slice, thereby enabling more reliable decoding of the attribute data.

[0127] <Attribute data encoding process flow> Next, an example of the flow of the attribute data encoding process executed in step S104 in FIG. 16 will be described with reference to the flowchart in FIG.

[0128] When the attribute data encoding process is started, in step S131, the layering unit 131 of the attribute data encoding unit 104 layers the attribute data so as to correspond to the layered structure of the geometry data.

[0129] In step S132, the slice structure generating unit 132 generates a slice structure for the attribute data.

[0130] In step S133, the neighboring point search unit 133 searches for neighboring points that refer to attribute data in order to derive a predicted value of the processing target point, based on the geometry data.

[0131] In this case, the neighboring point search unit 133 performs the neighboring point search by applying the present technology described above in <1. Restrictions on Neighboring Point Search>. For example, as described above in <Node Grouping of Attributes>, the neighboring point search unit 133 may perform the neighboring point search for setting a reference point to be referred to when deriving a predicted value of the attribute data of the node to be processed, targeting only nodes that are decoded before the node to be processed during decoding, among nodes in a tree structure in which slices, which are node groups that can be coded independently of each other, are formed.

[0132] The tree structure of the attribute data may have nodes that represent attribute data of each point hierarchically organized based on the geometry data. For example, the tree structure of the attribute data may have nodes that represent attribute data of each point hierarchically organized based on the geometry data so that a point also exists in a voxel one level higher than the voxel in which a voxel in the processing target level exists. Alternatively, the geometry data may form a hierarchical tree structure based on the resolution of the geometry, and each node in the tree structure of the attribute data may correspond to each node in the tree structure of the geometry data. Furthermore, slices, which are node groups that can be coded independently of each other, may be formed in the tree structure of the geometry data, and the structure of the slices formed in the tree structure of the attribute data may correspond to the structure of the slices formed in the tree structure of the geometry data.

[0133] Furthermore, as described above in <Inter Reference>, the neighboring point search unit 133 may perform a neighboring point search only on nodes that are higher in the hierarchy than the node to be processed in the tree structure of the attribute data and that are decoded before the node to be processed during decoding.

[0134] As a neighboring point search for the inter-referencing, the neighboring point search unit 133 may perform a neighboring point search only on nodes in the tree structure of the attribute data that belong to the same slice as the target node to be processed, are decoded before the target node during decoding, and are in a higher hierarchical level than the target node, as well as nodes that belong to a slice that is decoded before the slice to which the target node belongs during decoding, as described above in <Limiting neighboring point search targets for inter-referencing based on slice structure>.

[0135] Furthermore, as a neighbor point search for the inter-reference, the neighbor point search unit 133 may perform a neighbor point search by targeting only ancestor nodes of the node being processed among the nodes in the tree structure of the attribute data, as described above in <Limiting neighbor point search targets for inter-reference based on tree structure>.

[0136] Furthermore, as described above in <Intra Reference>, the neighboring point search unit 133 may perform a neighboring point search only for nodes at the same level as the node to be processed in the tree structure of the attribute data, which nodes will be decoded before the node to be processed during decoding.

[0137] As a neighboring point search for the inter-reference, the neighboring point search unit 133 may perform a neighboring point search only on nodes in the tree structure of the attribute data that belong to the same slice as the node to be processed, that are decoded before the node to be processed during decoding, and that are at the same hierarchical level as the node to be processed, as described above in <Limiting neighboring point search targets for intra-reference based on slice structure>.

[0138] In step S134, the prediction processing unit 134 derives a predicted value of the attribute data of the node to be processed based on the reference structure between nodes corresponding to the result of the neighbor point search in step S133, and derives a difference value between the attribute data and the predicted value.

[0139] In step S135, the quantization unit 135 quantizes the difference value derived in step S134.

[0140] In step S136, the encoding unit 136 encodes the difference value quantized in step S135.

[0141] When the process of step S136 ends, the attribute data encoding process ends.

[0142] By performing the processing at each step in this manner, the encoding device 100 (attribute data encoding unit 104) can encode the attribute data so that it can be decoded independently for each slice, thereby enabling more reliable decoding of the attribute data.

[0143] 4. Second Embodiment <Decryption device> Fig. 18 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which the present technology is applied. The decoding device 200 shown in Fig. 18 is a device that decodes encoded data of a point cloud (3D data). The present technology (for example, various methods described with reference to Figs. 1 to 13) can be applied to the decoding device 200.

[0144] Note that Fig. 18 shows the main processing units, data flows, etc., and does not necessarily show everything. That is, in the decoding device 200, there may be processing units that are not shown as blocks in Fig. 18, and there may be processing or data flows that are not shown as arrows, etc. in Fig. 18.

[0145] As shown in FIG. 18, the decoding device 200 includes a decoding target setting unit 201 , an encoded data extraction unit 202 , a geometry data decoding unit 203 , an attribute data decoding unit 204 , and a point cloud generation unit 205 .

[0146] The decoding target setting unit 201 performs processing related to setting layers (LoDs) and slices (node ​​groups) to be decoded. For example, the decoding target setting unit 201 sets layers and regions to be decoded, such as up to which layers to decode and which slices to decode, for the coded data of the point cloud held in the coded data extraction unit 202. The method of setting the layers and slices to be decoded is arbitrary.

[0147] For example, the decoding target setting unit 201 may set the layers and slices based on instructions related to the layers and slices supplied from an external source such as a user or an application. Alternatively, the decoding target setting unit 201 may determine and set the layers and slices to be decoded based on any information such as an output image.

[0148] For example, the decoding target setting unit 201 may set the layers and slices to be decoded based on the viewpoint position, direction, angle of view, viewpoint movement (movement, pan, tilt, zoom) etc. of the two-dimensional image generated from the point cloud.

[0149] The data unit for setting the decoding target may be any. For example, the decoding target setting unit 201 may set layers and slices for the entire point cloud. The decoding target setting unit 201 may also set layers and slices for each object. Furthermore, the decoding target setting unit 201 may also set layers and slices for each partial region within an object. Of course, layers and slices may also be set in data units other than these examples.

[0150] The coded data extraction unit 202 acquires and stores the bitstream input to the decoding device 200. The coded data extraction unit 202 extracts coded data of geometry data and attribute data corresponding to the decoding target range specified by the decoding target setting unit 201 from the bitstream it stores. The coded data extraction unit 202 supplies the coded data of the extracted geometry data to the geometry data decoding unit 203. The coded data extraction unit 202 supplies the coded data of the extracted attribute data to the attribute data decoding unit 204.

[0151] The geometry data decoding unit 203 acquires the coded data of the geometry data supplied from the coded data extraction unit 202. The geometry data decoding unit 203 decodes the coded data to generate geometry data (decoded result). This decoding method may be any method similar to that used by the geometry data decoding unit 102 of the encoding device 100. The geometry data decoding unit 203 supplies the generated geometry data (decoded result) to the attribute data decoding unit 204 and the point cloud generation unit 205.

[0152] The attribute data decoding unit 204 acquires the coded data of the attribute data supplied from the coded data extraction unit 202. The attribute data decoding unit 204 acquires the geometry data (decoded result) supplied from the geometry data decoding unit 203. The attribute data decoding unit 204 decodes the coded data using the geometry data (decoded result) to generate attribute data (decoded result). The attribute data decoding unit 204 supplies the generated attribute data (decoded result) to the point cloud generation unit 205.

[0153] The point cloud generation unit 205 acquires the geometry data (decoded result) supplied from the geometry data decoding unit 203. The point cloud generation unit 205 acquires the attribute data (decoded result) supplied from the attribute data decoding unit 204. The point cloud generation unit 205 associates the geometry data (decoded result) with the attribute data (decoded result) to generate point cloud data (decoded result). The point cloud generation unit 205 outputs the generated point cloud data (decoded result) to the outside of the decoding device 200.

[0154] In such a decoding device 200, the present technology described above in <1. Limitation of Neighboring Point Search> may be applied to the attribute data decoding unit 204. That is, in this case, the attribute data decoding unit 204 decodes the encoded data of the attribute data by a method to which the present technology described above in <1. Limitation of Neighboring Point Search> is applied.

[0155] With this configuration, the decoding device 200 can encode the attribute data so that it can be decoded independently for each slice, thereby more reliably decoding the attribute data.

[0156] Furthermore, the present technology described above in <2. Restrictions on Association> may be applied to the point cloud generation unit 205. That is, in this case, the point cloud generation unit 205 generates point cloud data by a method to which the present technology described above in <2. Restrictions on Association> is applied.

[0157] For example, the point cloud generation unit 205 may associate the attribute data of a point in the decoding target area restored by the attribute data decoding unit 204 (the restoration unit 236 described later) with the geometry data of that point, as described above in <Matching restrictions based on slice structure>.

[0158] Furthermore, as described above in <Removal after Correspondence>, the point cloud generation unit 205 may correspond the attribute data and geometry data of all points restored by the attribute data decoding unit 204 (the restoration unit 236 described later), and remove points in areas not to be decoded from among the points for which the attribute data and geometry data are associated.

[0159] Furthermore, as described above in <Removal before matching>, the point cloud generation unit 205 may remove the attribute data of points in the non-decoding target area restored by the attribute data decoding unit 204 (the restoration unit 236 described later), and match the attribute data of the points in the decoding target area with the geometry data.

[0160] With this configuration, the decoding device 200 can associate the geometry data with the attribute data. In other words, by doing so, the geometry data and the attribute data can be more reliably associated with each other, and point cloud data can be generated more reliably.

[0161] Note that these processing units (the decoding target setting unit 201 to the point cloud generation unit 205) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Furthermore, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using these. Of course, each processing unit may have both of these configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of the processing units may be independent of each other. For example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and by executing a program.

[0162] <Attribute data decoding part> Fig. 19 is a block diagram showing an example of the main configuration of the attribute data decoding unit 204 (Fig. 18). Note that Fig. 19 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, the attribute data decoding unit 204 may have processing units that are not shown as blocks in Fig. 19, or may have processing or data flows that are not shown as arrows, etc. in Fig. 19.

[0163] As shown in FIG. 19, the attribute data decoding unit 204 includes a decoding unit 231 , an inverse quantization unit 232 , a layering unit 233 , a slice structure generation unit 234 , a neighboring point search unit 235 , and a restoration unit 236 .

[0164] The decoding unit 231 executes processing related to decoding. For example, the decoding unit 231 acquires coded data of attribute data supplied from the coded data extraction unit 202 (FIG. 18). The decoding unit 231 decodes the coded data. This decoding obtains a difference value between the attribute data and its predicted value. Note that this difference value is quantized. Furthermore, this decoding method may be any method as long as it is compatible with the coding method used by the coding unit 136 (FIG. 15) of the coding device 100. The decoding unit 231 supplies the generated difference value (quantized difference value) to the inverse quantization unit 232.

[0165] The inverse quantization unit 232 obtains the quantized difference value supplied from the decoding unit 231. The inverse quantization unit 232 inverse quantizes the quantized difference value to derive a difference value. The inverse quantization unit 232 supplies the difference value to the layering unit 233.

[0166] The layering unit 233 acquires the difference values ​​supplied from the inverse quantization unit 232. The layering unit 233 layers (structures) the attribute data based on the geometry data supplied from the geometry data decoding unit 203. As a result, the layering unit 233 forms a tree structure for the attribute data similar to that in the encoding device 100. The layering unit 233 supplies the layered difference values ​​to the slice structure generation unit 234.

[0167] The slice structure generation unit 234 acquires the layered difference values ​​(attribute data) supplied from the layering unit 233. The slice structure generation unit 234 performs slice division on the layered difference values ​​(attribute data) in the same manner as the slice structure generation unit 132, to generate a slice structure. At this time, the slice structure generation unit 234 performs slice division so as to divide each node into regions using geometry data (that is, based on the geometry of the points). For example, the slice structure generation unit 234 generates a slice structure of the difference values ​​(attribute data) so as to have the same slice structure as the geometry data. The slice structure generation unit 234 supplies the difference values ​​(attribute data) for which the slice structure has been generated to the neighboring point search unit 235 together with the geometry data.

[0168] The neighboring point search unit 235 executes processing related to searching for neighboring points by referencing attribute data in order to derive a predicted value of the processing target point. For example, the neighboring point search unit 235 acquires difference values ​​(attribute data) and geometry data supplied from the slice structure generation unit 234. The neighboring point search unit 235 also executes a neighboring point search based on the geometry data.

[0169] The neighboring point search unit 235 performs the neighboring point search by applying the present technology described above in <1. Restrictions on the neighboring point search>. For example, as described above in <Node grouping of attributes>, the neighboring point search unit 235 may perform the neighboring point search for setting a reference point to be referenced when deriving a predicted value of a processing target node, targeting only nodes decoded by the decoding unit 231 before a processing target node within a tree structure in which slices, which are node groups that can be coded independently of each other, are formed, with attribute data of each point of a point cloud as the node.

[0170] The tree structure of the attribute data may have nodes that represent attribute data of each point hierarchically organized based on the geometry data. For example, the tree structure of the attribute data may have nodes that represent attribute data of each point hierarchically organized based on the geometry data so that a point also exists in a voxel one level higher than the voxel in which a voxel in the processing target level exists. Alternatively, the geometry data may form a hierarchical tree structure based on the resolution of the geometry, and each node in the tree structure of the attribute data may correspond to each node in the tree structure of the geometry data. Furthermore, slices, which are node groups that can be coded independently of each other, may be formed in the tree structure of the geometry data, and the structure of the slices formed in the tree structure of the attribute data may correspond to the structure of the slices formed in the tree structure of the geometry data.

[0171] By matching the tree structure of the attribute data to the tree structure of the geometry data in this way, scalable decoding of the point cloud data can be realized. Also, by matching the slice structure of the attribute data to the slice structure of the geometry data, the point cloud data can be restored independently for each slice.

[0172] Furthermore, as described above in <Inter Reference>, the neighbor point search unit 235 may perform a neighbor point search only on nodes in the tree structure of the attribute data that are at a higher level than the node to be processed and that have been decoded by the decoding unit 231 before the node to be processed.

[0173] As a neighbor point search for the inter-referencing, the neighbor point search unit 235 may perform a neighbor point search only on nodes in the tree structure of the attribute data that belong to the same slice as the node to be processed, that are in a higher hierarchical level than the node to be processed, and that have been decoded by the decoding unit 231 before the node to be processed, as described above in <Limitation on neighbor point search targets for inter-referencing based on slice structure>, and that belong to slices that have been decoded by the decoding unit 231 before the slice to which the node to be processed belongs.

[0174] Furthermore, as a neighbor point search for the inter-reference, the neighbor point search unit 235 may perform a neighbor point search by targeting only ancestor nodes of the node being processed among the nodes in the tree structure of the attribute data, as described above in <Limiting neighbor point search targets for inter-reference based on tree structure>.

[0175] Furthermore, as described above in <Intra Reference>, the neighboring point search unit 235 may perform a neighboring point search only on nodes at the same level as the node to be processed in the tree structure of the attribute data that have been decoded by the decoding unit 231 before the node to be processed.

[0176] As a neighboring point search for the inter-reference, the neighboring point search unit 235 may perform a neighboring point search only on nodes in the tree structure of the attribute data that belong to the same slice as the node to be processed, that have been decoded by the decoding unit 231 before the node to be processed, and that are at the same hierarchical level as the node to be processed, as described above in <Limiting neighboring point search targets for intra-reference based on slice structure>.

[0177] The neighboring point search unit 235 supplies the search results to the restoration unit 236 together with the attribute data and geometry data.

[0178] The restoration unit 236 executes processing related to the restoration of attribute data. For example, the restoration unit 236 acquires the search results of the neighboring point search, the difference values ​​(attribute data), and the geometry data supplied from the neighboring point search unit 235. The restoration unit 236 uses this information to derive predicted values ​​of the attribute data of the node to be processed. For example, the restoration unit 236 sets the searched neighboring points as a parent node or a grandparent node, and derives predicted values ​​of the attribute data of the node to be processed using the attribute data of these nodes.

[0179] Furthermore, the restoration unit 236 adds the derived predicted value to the difference value for the node to be processed to restore the attribute data, and then supplies the derived attribute data to the point cloud generation unit 205.

[0180] As described above, the neighboring point search unit 235 performs a neighboring point search by applying the present technology, and the decoding device 200 (the attribute data decoding unit 204) can decode the attribute data for each slice independently. Therefore, the attribute data can be decoded more reliably.

[0181] These processing units (the decoding unit 231 to the restoration unit 236) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Furthermore, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-described processing using a logic circuit and the other part by executing a program. The configurations of the processing units may be independent of each other. For example, some processing units may realize part of the above-described processing using a logic circuit, other processing units may execute a program to realize the above-described processing, and still other processing units may realize the above-described processing using both a logic circuit and by executing a program.

[0182] <Decryption process flow> Next, a description will be given of the processing executed by the decoding device 200. The decoding device 200 decodes the encoded data of the point cloud by executing a decoding process. An example of the flow of this decoding process will be described with reference to the flowchart in FIG.

[0183] When the decoding process starts, the decoding target setting unit 201 of the decoding device 200 sets the LoD and slices to be decoded in step S201.

[0184] In step S202, the coded data extraction unit 202 acquires and holds the bitstream, and extracts coded data of the LoD and the geometry data and attribute data (difference values) of the slice (that is, the decoding target) set in step S201.

[0185] In step S203, the geometry data decoding unit 203 decodes the coded data extracted in step S202 to generate geometry data (decoded result).

[0186] In step S204, the attribute data decoding unit 204 executes attribute data decoding processing to decode the coded data extracted in step S202 and generate a difference value (attribute data).

[0187] In step S205, the point cloud generation unit 205 executes a point cloud generation process and generates a point cloud (decoding result) by associating the geometry data generated in step S203 with the difference value (attribute data) generated in step S204.

[0188] When the process of step S205 ends, the decoding process ends.

[0189] The present technology described above in <1. Limitation of Neighboring Point Search> can be applied to the attribute data decoding process executed in step S204 of such a decoding process. That is, in this case, the attribute data decoding unit 204 executes the attribute data decoding process by the method to which the present technology described above in <1. Limitation of Neighboring Point Search> is applied, and decodes the encoded data of the difference value (attribute data).

[0190] Furthermore, the present technology described above in <2. Restrictions on Association> may be applied to the point cloud generation process executed in step S205. That is, in this case, the point cloud generation unit 205 executes the point cloud generation process by a method to which the present technology described above in <1. Restrictions on Neighbor Point Search> is applied, and associates attribute data of points in the restored decoding target region with geometry data of the points, thereby generating point cloud data.

[0191] By performing the processing of each step in this manner, the decoding device 200 can decode the attribute data for each slice independently. Also, the decoding device 200 can associate the geometry data with the attribute data. Therefore, the attribute data can be decoded more reliably.

[0192] <Attribute data decoding process flow> Next, an example of the flow of the attribute data decoding process executed in step S204 of FIG. 20 will be described with reference to the flowchart of FIG.

[0193] When the attribute data decoding process starts, in step S231, the decoding unit 231 of the attribute data decoding unit 204 decodes the coded data of the attribute data (difference value) to generate a difference value. This difference value is quantized.

[0194] In step S232, the inverse quantization unit 232 inverse quantizes the quantized difference value obtained in step S231 to obtain a difference value.

[0195] In step S233, the layering unit 233 layers the difference values ​​(attribute data) obtained in step S232. For example, the layering unit 233 layers the difference values ​​so as to correspond to the hierarchical structure of the geometry data.

[0196] In step S234, the slice structure generation unit 234 performs slice division on the difference values ​​(attribute data) layered in step S233, in the same manner as the slice structure generation unit 132, to generate a slice structure. At this time, the slice structure generation unit 234 performs slice division so as to divide each node into regions using geometry data (that is, based on the geometry of the points). For example, the slice structure generation unit 234 generates a slice structure for the difference values ​​(attribute data) so as to have the same slice structure as the geometry data.

[0197] In step S235, the neighboring point search unit 235 searches for neighboring points by referring to attribute data in order to derive a predicted value of the processing target point.

[0198] In this case, the neighboring point search unit 235 performs the neighboring point search by applying the present technology described above in <1. Restrictions on Neighboring Point Search>. For example, as described above in <Node Grouping of Attributes>, the neighboring point search unit 235 may perform the neighboring point search for setting a reference point to be referred to when deriving a predicted value of a processing target node, targeting only nodes that have been decoded by the decoding unit 231 before the processing target node, within nodes of a tree structure in which slices, which are node groups that can be coded independently of each other, are formed, with attribute data of each point of the point cloud as nodes.

[0199] The tree structure of the attribute data may have nodes that represent attribute data of each point hierarchically organized based on the geometry data. For example, the tree structure of the attribute data may have nodes that represent attribute data of each point hierarchically organized based on the geometry data so that a point also exists in a voxel one level higher than the voxel in which a voxel in the processing target level exists. Alternatively, the geometry data may form a hierarchical tree structure based on the resolution of the geometry, and each node in the tree structure of the attribute data may correspond to each node in the tree structure of the geometry data. Furthermore, slices, which are node groups that can be coded independently of each other, may be formed in the tree structure of the geometry data, and the structure of the slices formed in the tree structure of the attribute data may correspond to the structure of the slices formed in the tree structure of the geometry data.

[0200] Furthermore, as described above in <Inter Reference>, the neighbor point search unit 235 may perform a neighbor point search only on nodes at a higher level in the tree structure of the attribute data than the node to be processed, which nodes have been decoded by the decoding unit 231 before the node to be processed.

[0201] As a neighbor point search for the inter-referencing, the neighbor point search unit 235 may perform a neighbor point search only on nodes in the tree structure of the attribute data that belong to the same slice as the node to be processed, that are in a higher hierarchical level than the node to be processed, and that have been decoded by the decoding unit 231 before the node to be processed, as described above in <Limitation on neighbor point search targets for inter-referencing based on slice structure>, and that belong to slices that have been decoded by the decoding unit 231 before the slice to which the node to be processed belongs.

[0202] Furthermore, as a neighbor point search for the inter-reference, the neighbor point search unit 235 may perform a neighbor point search by targeting only ancestor nodes of the node being processed among the nodes in the tree structure of the attribute data, as described above in <Limiting neighbor point search targets for inter-reference based on tree structure>.

[0203] Furthermore, as described above in <Intra Reference>, the neighboring point search unit 235 may perform a neighboring point search only on nodes at the same level as the node to be processed in the tree structure of the attribute data that have been decoded by the decoding unit 231 before the node to be processed.

[0204] As a neighboring point search for the inter-reference, the neighboring point search unit 235 may perform a neighboring point search only on nodes in the tree structure of the attribute data that belong to the same slice as the node to be processed, that have been decoded by the decoding unit 231 before the node to be processed, and that are at the same hierarchical level as the node to be processed, as described above in <Limiting neighboring point search targets for intra-reference based on slice structure>.

[0205] In step S236, the restoration unit 236 derives a predicted value of the attribute data of the node to be processed using the search results of the neighboring point search executed in step S235, the difference value (attribute data) obtained in step S232, and geometry data. For example, the restoration unit 236 sets the searched neighboring points as a parent node or a grandparent node, and derives a predicted value of the attribute data of the node to be processed using the attribute data of those nodes. Furthermore, the restoration unit 236 adds the derived predicted value to the difference value for the node to be processed, thereby restoring the attribute data.

[0206] When the process of step S236 ends, the attribute data decoding process ends.

[0207] By performing the processing of each step in this manner, the decoding device 200 (the attribute data decoding unit 204) can decode the attribute data for each slice independently, thereby more reliably decoding the attribute data.

[0208] <Point cloud generation process flow 1> Next, an example of the flow of the point cloud generation process executed in step S205 in Fig. 20 will be described with reference to the flowchart in Fig. 22. Note that this flowchart corresponds to the example described in <Removal after association> in <2. Association restrictions>.

[0209] When the point cloud generation process is started, in step S251, the point cloud generation unit 205 associates geometry data with attribute data, including regions that are not to be output.

[0210] Then, in step S252, the point cloud generating unit 205 deletes points in areas that are not to be output based on the geometry data.

[0211] When the process of step S252 ends, the process returns to FIG.

[0212] In this way, the point cloud generation unit 205 can remove points in the non-decoding target area and leave points in the decoding target area, as described above in <Removal after Correspondence>. Therefore, the decoding device 200 (point cloud generation unit 205) can more reliably associate the geometry data with the attribute data, and more reliably generate point cloud data.

[0213] <Point cloud generation process flow 2> Next, another example of the flow of the point cloud generation process executed in step S205 of Fig. 20 will be described with reference to the flowchart of Fig. 23. Note that this flowchart corresponds to the example described in <Removal before association> of <2. Association restrictions>.

[0214] When the point cloud generation process is started, the point cloud generation unit 205 deletes attribute data of regions that are not to be output based on low-resolution geometry data in step S271.

[0215] In step S272, the point cloud generating unit 205 associates the geometry data and attribute data of the output target region.

[0216] When the process of step S272 ends, the process returns to FIG.

[0217] By doing so, the point cloud generating unit 205 can associate the geometry data with the attribute data as described above in <Removal before association>. In other words, by doing so, the decoding device 200 (point cloud generating unit 205) can more reliably associate the geometry data with the attribute data, and more reliably generate point cloud data.

[0218] <5. Notes> <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0219] FIG. 24 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0220] In a computer 900 shown in FIG. 24, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0221] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0222] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0223] In a computer configured as above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0224] The program executed by the computer can be applied by recording it on removable media 921 such as package media, for example. In this case, the program can be installed in storage unit 913 via input / output interface 910 by inserting removable media 921 into drive 915.

[0225] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0226] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0227] <Applicable targets of this technology> Although the above describes the application of this technology to the encoding and decoding of point cloud data, this technology is not limited to these examples and can be applied to the encoding and decoding of 3D data of any standard. For example, when encoding or decoding mesh data, the mesh data may be converted into point cloud data, and the encoding and decoding may be performed using this technology. In other words, as long as it does not conflict with the above-described technology, various processes such as encoding and decoding methods, and specifications of various data such as 3D data and metadata, are arbitrary. Furthermore, as long as it does not conflict with the above-described technology, some of the above-described processes and specifications may be omitted.

[0228] The present technology can be applied to any configuration, for example, to transmitters and receivers (e.g., television receivers and mobile phones) in satellite broadcasting, cable TV and other wired broadcasting, distribution over the Internet, and distribution to terminals via cellular communication, as well as various electronic devices such as devices that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media (e.g., hard disk recorders and cameras).

[0229] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0230] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, AV (Audio Visual) equipment, a portable information processing terminal, or an IoT (Internet of Things) device.

[0231] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0232] <Fields and uses where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, their uses are also arbitrary.

[0233] <Other> The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0234] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0235] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0236] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0237] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0238] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0239] The present technology can also be configured as follows. (1) A neighboring point search unit that performs a neighboring point search for setting a reference point to be referenced when deriving a predicted value of the attribute data of a target node, targeting only the nodes that are decoded before a target node during decoding, among the nodes in a tree structure in which slices are formed, each of which is a node group that can be encoded independently of each other, and which represents attribute data of each point of a point cloud that represents a three-dimensional object as a set of points; an encoding unit that encodes a difference value between the attribute data of the processing target node and the predicted value derived using the attribute data of the reference point set by the neighboring point search unit through the neighboring point search; An information processing device comprising: (2) The neighboring point search unit performs the neighboring point search only on the nodes that are decoded before the processing target node during the decoding, among the nodes in a higher hierarchy than the processing target node in the tree structure. The information processing device described in (1). (3) The neighboring point search unit is configured to search for a neighboring point among the nodes in the tree structure. The node in a higher layer than the processing target node belongs to the same slice as the processing target node and is decoded before the processing target node during the decoding; the node belonging to a slice that is decoded earlier than the slice to which the processing target node belongs during the decoding; The neighboring point search is performed only on (2) An information processing device according to the present invention. (4) The neighboring point search unit performs the neighboring point search only on ancestor nodes of the target node among the nodes in the tree structure. (2) An information processing device according to the present invention. (5) The neighboring point search unit performs the neighboring point search only on the nodes that are decoded earlier than the processing target node during the decoding, among the nodes in the same layer as the processing target node in the tree structure. The information processing device described in (1). (6) The neighboring point search unit performs the neighboring point search only on the nodes in the tree structure that belong to the same slice as the target node, are decoded before the target node during the decoding, and are in the same layer as the target node. (5) An information processing device according to the present invention. (7) The tree structure has the attribute data of each point hierarchically arranged based on geometry data as the nodes. An information processing device according to any one of (1) to (6). (8) The tree structure is based on the geometry data and has the attribute data of each point hierarchically arranged so that the point also exists in the voxel of the next higher layer to which the voxel in which the point in the processing layer exists belongs, as the node. (7) An information processing device according to (7). (9) The geometry data forms a hierarchical tree structure based on the resolution of the geometry; each node of the tree structure of the attribute data corresponds to each node of the tree structure of the geometry data; The tree structure of the geometry data includes slices, which are node groups that can be coded independently of each other; The structure of the slices formed in the tree structure of the attribute data corresponds to the structure of the slices formed in the tree structure of the geometry data. (8) An information processing device according to (8). (10) A node is defined as attribute data of each point of a point cloud that represents a three-dimensional object as a set of points, and among the nodes in a tree structure in which slices are formed, which are node groups that can be coded independently of each other, a neighboring point search is performed for only the nodes that are decoded before the processing target node during decoding, in order to set a reference point to be referenced when deriving a predicted value of the attribute data of the processing target node; A difference value between the attribute data of the processing target node and the predicted value derived using the attribute data of the reference point set by the neighbor point search is encoded. Information processing methods.

[0240] (11) A decoding unit that decodes coded data in which a difference value between attribute data of each point of a point cloud that represents a three-dimensional object as a set of points is coded and a predicted value of the attribute data of a processing target node in a tree structure in which slices are formed as node groups that can be coded independently of each other, and derives the difference value of the processing target node; a neighboring point search unit that performs a neighboring point search for setting a reference point to be referenced when deriving the predicted value of the processing target node, with only the nodes in the tree structure that have been decoded by the decoding unit before the processing target node; and a restoration unit that restores the attribute data of the processing target node by adding the difference value derived by the decoding unit and the predicted value derived using the attribute data of the reference point set by the neighbor point search unit through the neighbor point search; an association unit that associates the attribute data of the point in the decoding target area restored by the restoration unit with geometry data of the point; An information processing device comprising: (12) The neighboring point search unit is configured to search for a neighboring point among the nodes in the tree structure. The node in a higher layer than the processing target node belongs to the same slice as the processing target node and is decoded by the decoding unit before the processing target node; The node belonging to the slice that was decoded by the decoding unit earlier than the slice to which the processing target node belongs, and The neighboring point search is performed only on (11) An information processing device according to (11). (13) The neighboring point search unit performs the neighboring point search only on ancestor nodes of the target node among the nodes in the tree structure. (11) An information processing device according to (11). (14) The neighboring point search unit performs the neighboring point search only on the nodes in the tree structure that belong to the same slice as the processing target node, that are decoded by the decoding unit before the processing target node, and that are at the same layer as the processing target node. (11) An information processing device according to (11). (15) The tree structure has the attribute data of each point hierarchically arranged based on geometry data as a node. An information processing device according to any one of (11) to (14). (16) The tree structure is based on the geometry data and has the attribute data of each point hierarchically arranged so that the point also exists in the voxel of the next higher layer to which the voxel in which the point in the processing layer exists belongs, as the node. (15) An information processing device according to (15). (17) The geometry data forms a hierarchical tree structure based on the resolution of the geometry; each node of the tree structure of the attribute data corresponds to each node of the tree structure of the geometry data; The tree structure of the geometry data includes slices, which are node groups that can be coded independently of each other; The structure of the slices formed in the tree structure of the attribute data corresponds to the structure of the slices formed in the tree structure of the geometry data. (16) An information processing device according to (16). (18) The association unit Associating the attribute data of all the points restored by the restoration unit with the geometry data; The points in the non-decoding region in which the attribute data and the geometry data are associated are removed. (17) An information processing device according to (17). (19) The association unit removing the attribute data of the points in the non-decoding target region restored by the restoration unit; The attribute data of the points in the decoding target area is associated with the geometry data. (17) An information processing device according to (17). (20) A method for decoding coded data in which attribute data of each point of a point cloud representing a three-dimensional object as a set of points is defined as a node, and a difference value between the attribute data and a predicted value of the attribute data of a processing target node in a tree structure in which slices are formed as node groups that can be coded independently of each other is coded, and deriving the difference value of the processing target node; performing a neighborhood point search for setting a reference point to be referenced when deriving the predicted value of the processing target node, with only the nodes in the tree structure that have been decoded earlier than the processing target node being targeted; restoring the attribute data of the processing target node by adding the derived difference value and the predicted value derived using the attribute data of the reference point set by the neighbor point search; The attribute data of the point in the restored decoding target region is associated with the geometry data of the point. Information processing methods. [Explanation of symbols]

[0241] 100 encoding device, 101 geometry data encoding unit, 102 geometry data decoding unit, 103 point cloud generation unit, 104 attribute data encoding unit, 105 bitstream generation unit, 131 layering unit, 132 slice structure generation unit, 133 neighbor point search unit, 134 prediction processing unit, 135 quantization unit, 136 encoding unit, 200 decoding device, 201 decoding target setting unit, 202 encoded data extraction unit, 203 geometry data decoding unit, 204 attribute data decoding unit, 205 point cloud generation unit, 231 decoding unit, 232 inverse quantization unit, 233 layering unit, 234 slice structure generation unit, 235 neighbor point search unit, 236 restoration unit, 900 computer

Claims

1. a hierarchical structure generating unit that associates nodes of attribute data of a point cloud with a tree structure of geometry data of the point cloud, the tree structure representing a three-dimensional object as a set of points; a slice structure generating unit that divides the attribute data to generate a slice structure including a first slice and a second slice; a neighboring point search unit that performs a neighboring point search for a point that is decoded before a processing target point during decoding, in order to set a reference point that is to be referenced when deriving a predicted value of attribute data of the processing target point; an encoding unit that encodes a difference value between the attribute data of the processing target point and the predicted value derived using the reference point set by the neighboring point search unit through the neighboring point search; Equipped with The neighboring point search unit excludes, from the reference points, nodes in the second slice that do not include an ancestor node of the node to be processed in the inter-reference of the node to be processed in the first slice. Encoding device.

2. The slice structure includes a third slice that includes a node higher than a node of the first slice in the tree structure and is decoded prior to the first slice during decoding; The neighboring point search unit performs the neighboring point search on a node in a higher layer than the processing target node belonging to the first slice and a node belonging to the third slice. The encoding device according to claim 1 .

3. The neighboring point search unit performs the neighboring point search on a node in the tree structure that is at the same layer as the processing target node and that is to be decoded prior to the processing target node during the decoding. The encoding device according to claim 1 .

4. The neighboring point search unit performs the neighboring point search on a node in the same layer as the processing target node, which belongs to the first slice, is decoded before the processing target node during the decoding, and is a target. The encoding device according to claim 3 .

5. The tree structure has the attribute data of each point hierarchically arranged based on geometry data as a node. The encoding device according to claim 1 .

6. The tree structure is based on the geometry data and has the attribute data of each point hierarchically arranged so that a point also exists in a voxel one layer higher than the voxel in which the point in the processing target layer exists. The encoding device according to claim 5 .

7. For a point cloud that represents a three-dimensional object as a set of points, a node of attribute data of the point cloud is associated with a tree structure of geometry data of the point cloud; Dividing the attribute data to generate a slice structure including a first slice and a second slice; performing a neighboring point search for a point that is decoded before the processing target point during decoding, in order to set a reference point to be referred to when deriving a predicted value of attribute data of the processing target point; encoding a difference value between the attribute data of the processing target point and the predicted value derived using the reference point set by the neighbor point search; In the neighbor point search, in the inter-reference of the processing target node in the first slice, a node in the second slice that does not include an ancestor node of the processing target node is excluded from the reference points. Encoding method.

8. a decoding unit that decodes encoded data in which a difference value between attribute data of a processing target point and a predicted value of the attribute data is encoded for a point cloud that represents a three-dimensional object as a set of points, and derives the difference value of the processing target point; a hierarchical structure generating unit that associates nodes of attribute data of the point cloud with a tree structure of geometry data of the point cloud; a slice structure generating unit that divides the attribute data to generate a slice structure including a first slice and a second slice; a neighboring point search unit that performs a neighboring point search for a point that has been decoded by the decoding unit before the processing target point, in order to set a reference point to be referenced when deriving the predicted value of the processing target point; a restoration unit that restores the attribute data of the processing target point by adding the difference value derived by the decoding unit and the predicted value derived using the reference point set by the neighboring point search unit through the neighboring point search; an association unit that associates the attribute data of the points in the decoding target area restored by the restoration unit with geometry data of the points; Equipped with The neighboring point search unit excludes, from the reference points, nodes in the second slice that do not include an ancestor node of the node to be processed in the inter-reference of the node to be processed in the first slice. Decryption device.

9. the slice structure includes a third slice that includes a node higher than a node of the first slice in the tree structure and that has been decoded earlier than the first slice; The neighboring point search unit performs the neighboring point search on a node in a higher layer than the processing target node belonging to the first slice and a node belonging to the third slice. The decoding device according to claim 8.

10. The neighboring point search unit performs the neighboring point search on a node in the tree structure that belongs to the same slice as the processing target node, that is decoded by the decoding unit before the processing target node, and that is at the same hierarchical level as the processing target node. The decoding device according to claim 8.

11. The tree structure has the attribute data of each point hierarchically arranged based on geometry data as a node. The decoding device according to claim 8.

12. The tree structure is based on geometry data and has the attribute data of each point hierarchically arranged so that a point also exists in a voxel one layer higher than the voxel in which the point in the processing target layer exists. The decoding device according to claim 8.

13. The association unit Associating the attribute data and the geometry data of all points restored by the restoration unit; Remove points in the non-decoding region where the attribute data and the geometry data are associated with each other. The decoding device according to claim 8.

14. The association unit removing the attribute data of the points in the non-decoding target region restored by the restoration unit; The attribute data of the points in the decoding target area is associated with the geometry data. The decoding device according to claim 8.

15. For a point cloud that represents a three-dimensional object as a set of points, decoding encoded data in which a difference value between attribute data of a processing target point and a predicted value of the attribute data is encoded, and deriving the difference value of the processing target point; Associating nodes of attribute data of the point cloud with a tree structure of geometry data of the point cloud; Dividing the attribute data to generate a slice structure including a first slice and a second slice; performing a neighboring point search for a point decoded before the processing target point to set a reference point to be referenced when deriving the predicted value of the processing target point; restoring the attribute data of the processing target point by adding the derived difference value and the predicted value derived using the reference point set by the neighbor point search; Associating the attribute data of the restored point in the decoding target region with geometry data of the point; In the neighbor point search, in the inter-reference of the processing target node in the first slice, a node in the second slice that does not include an ancestor node of the processing target node is excluded from the reference points. Decryption method.

Images