Information processing apparatus and information processing method

MYPI2025006346A0Undetermined Publication Date: 2024-10-19SONY GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025006346
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-04-19
Filing Date
2024-04-02
Publication Date
2024-10-19

AI Technical Summary

Technical Problem

Conventional mesh decoding methods, particularly V-DMC, face challenges in parallelizing processing due to sequential subdivision methods, leading to increased processing times.

Method used

An information processing device and method that decodes a bitstream to generate a base mesh and independently calculates edges and vertices for each layer, using recursive generation processes to subdivide the mesh to a desired level of detail, allowing for parallel execution of element processes.

Benefits of technology

This approach effectively suppresses the increase in processing time for mesh decoding by enabling parallel processing of element tasks, thereby improving efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025006346_DRAWING_001
    Figure PI2025006346_DRAWING_001
  • Figure PI2025006346_DRAWING_002
    Figure PI2025006346_DRAWING_002
  • Figure PI2025006346_DRAWING_003
    Figure PI2025006346_DRAWING_003
Patent Text Reader

Abstract

The present disclosure relates to an information processing apparatus and an information processing method that are capable of suppressing an increase of a processing time for decoding a mesh. A bit stream is decoded to generate a base mesh; the base mesh is subdivided until a desired definition is obtained, the subdivision being performed by recursively repeating a first generation process, a second generation process, and a third generation process until a desired level is reached, the first generation process being a process for generating edges and vertices at a processing-target level of a definition independently for each edge at a next lower level of the definition, the second generation process being a process for generating the edges and vertices at the processing-target level with respect to a central triangle at the next lower level, the third generation process being a process for generating triangles at the processing-target level independently for each triangle at the next lower level; and displacement vectors are applied to division points that are vertices of the base mesh subdivided until the desired definition is obtained. The present disclosure can be applied to, for example, an information processing apparatus, an electronic apparatus, an information processing method, or a program. Figure 13
Need to check novelty before this filing date? Find Prior Art

Description

Information processing device and method

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that can suppress an increase in processing time for decoding meshes.

[0002] Conventionally, V-DMC (Video-based Dynamic Mesh Coding) has been used as a method for encoding meshes, which are 3D data that represent the three-dimensional structure of an object using vertices and connections (see, for example, Non-Patent Document 1). In V-DMC, the mesh to be encoded (original mesh) is represented by a coarse (i.e., low-resolution) base mesh and displacement vectors at division points obtained by subdividing the base mesh, and the base mesh and displacement vectors are encoded. The displacement vectors are stored (packed) in a two-dimensional image and encoded as an image. Because objects can change over time (are dynamic), the mesh (i.e., the base mesh and displacement vectors) are also dynamic. Therefore, the displacement vectors are encoded as a moving image (displacement video) in which the two-dimensional images are used as frames.

[0003] During decoding, the bitstream is decoded using a decoding method corresponding to the encoding method, and a base mesh and displacement vectors are restored (generated).The base mesh is then subdivided, and displacement vectors are applied to each division point to restore (generate) a mesh equivalent to the original mesh.

[0004] Khaled Mammou, Jungsun Kim, Alexis Tourapis, Dimitri Podborski, Krasimir Kolarov, "[V-CG] Apple's Dynamic Mesh Coding CfP Response", ISO / IEC JTC 1 / SC 29 / WG 7 m59281, April 2022

[0005] However, conventional base mesh subdivision methods involve sequential processing, making parallelization of the processing difficult and increasing the processing time.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress an increase in the processing time for decoding meshes.

[0007] An information processing device according to one aspect of the present technology includes: a decoding unit that decodes a bitstream and generates a base mesh; a subdivision unit that subdivides the base mesh to a desired level of fineness by recursively repeating, up to a desired level, a first generation process that generates edges and vertices of a layer of fineness to be processed independently for each edge of the layer one level lower in fineness; a second generation process that generates edges and vertices of the layer of fineness to be processed for a central triangle of the layer one level lower in fineness; and a third generation process that generates triangles of the layer of fineness to be processed independently for each triangle of the layer one level lower in fineness; and a displacement vector application unit that applies displacement vectors to division points, which are vertices of the base mesh subdivided to the desired level of fineness, wherein the base mesh is a mesh of lower fineness than the original mesh that is generated by thinning out vertices from an original mesh to be encoded that is composed of vertices and connections that represent the three-dimensional structure of an object, and the displacement vector indicates a difference in position between the division points and the vertices of the original mesh.

[0008] An information processing method according to one aspect of the present technology includes: decoding a bitstream; generating a base mesh; and recursively repeating, up to a desired layer, a first generation process for generating edges and vertices of a layer of resolution to be processed independently for each edge of the layer one level lower in resolution; a second generation process for generating edges and vertices of the layer of resolution to be processed for a central triangle in the layer one level lower; and a third generation process for generating triangles of the layer of resolution to be processed independently for each triangle in the layer one level lower, thereby subdividing the base mesh to a desired level of resolution, and applying displacement vectors to division points, which are vertices of the base mesh subdivided to the desired level of resolution; the base mesh is a mesh of lower resolution than the original mesh, generated by thinning out vertices from an original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object; and the displacement vector indicates a difference in position between the division points and the vertices of the original mesh.

[0009] In an information processing device and method according to one aspect of the present technology, a bitstream is decoded, a base mesh is generated, and a first generation process is performed to generate edges and vertices of the layer of resolution to be processed independently for each edge of the layer one level lower in resolution, a second generation process is performed to generate edges and vertices of the layer of resolution to be processed for the central triangle of the layer one level lower, and a third generation process is performed to generate triangles of the layer of resolution to be processed independently for each triangle of the layer one level lower, all of which are repeated recursively up to the desired layer, thereby subdividing the base mesh to the desired level of resolution, and applying displacement vectors to division points, which are vertices of the base mesh subdivided to the desired level of resolution.

[0010] FIG. 1 is a diagram illustrating a mesh. FIG. 2 is a diagram illustrating V-DMC. FIG. 3 is a diagram illustrating subdivision processing. FIG. 4 is a diagram illustrating subdivision processing. FIG. 5 is a diagram illustrating subdivision processing. FIG. 6 is a diagram illustrating subdivision processing. FIG. 7 is a diagram illustrating subdivision processing. FIG. 8 is a diagram illustrating subdivision processing. FIG. 9 is a diagram illustrating subdivision processing. FIG. 10 is a block diagram showing an example of the main configuration of a decoding device. FIG. 11 is a diagram illustrating updating of edges and vertices. FIG. 12 is a flowchart illustrating an example of the flow of decoding processing. FIG. 13 is a flowchart illustrating an example of the flow of subdivision processing. FIG. 14 is a diagram illustrating first generation processing. FIG. 15 is a diagram illustrating second generation processing. FIG. 16 is a diagram illustrating third generation processing. FIG. 17 is a diagram illustrating parallelization of processing. FIG. 18 is a block diagram showing an example of the main configuration of a computer.

[0011] Hereinafter, a mode for carrying out the present disclosure (hereinafter referred to as an embodiment) will be described. The description will be made in the following order: 1. Literature, etc. supporting technical content and technical terminology 2. Subdivision processing 3. Independence of element processing in subdivision processing 4. Supplementary notes

[0012] <1. Literature, etc. supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the content described in the embodiments, but also the content described in the following non-patent documents, etc. that were publicly known at the time of filing, and the content of other documents referenced in the following non-patent documents.

[0013] Non-patent document 1: (mentioned above)

[0014] In other words, the contents of the above-mentioned non-patent documents and the contents of other documents referenced in the above-mentioned non-patent documents are also used as the basis for determining the support requirements.

[0015] <2. Subdivision Processing> <V-DMC> Conventionally, 3D data representing the three-dimensional structure of a three-dimensional structure (an object with a three-dimensional shape) has been available as a mesh, which represents the three-dimensional shape of the object surface by forming polygons using vertices and connections (also called edges).

[0016] As shown in the upper left of Figure 1, in a mesh, vertices 11 and connections 12 connecting these vertices 11 form polygonal planes (polygons). These polygons (also called faces) represent the surface of a three-dimensional object, i.e., the three-dimensional shape of the object. A texture 13 can be applied to each face of this mesh.

[0017] Mesh data is composed of information such as that shown in the lower part of Figure 1. Vertex information 14, shown first from the left in the lower part of Figure 1, is information indicating the three-dimensional position (three-dimensional coordinates (X, Y, Z)) of each vertex 11 that constitutes the mesh. Connection information 15, shown second from the left in the lower part of Figure 1, is information indicating each connection (edge) 12 that constitutes the mesh. Texture image 16, shown third from the left in the lower part of Figure 1, is map information for texture 13 that is applied to each face. UV map 17, shown fourth from the left in the lower part of Figure 1, is information indicating the correspondence between vertices 11 and texture 13. The UV map 17 indicates the coordinates (UV coordinates) of each vertex 11 in the texture image 16.

[0018] As an example of such a mesh coding method, there is V-DMC (Video-based Dynamic Mesh Coding) as disclosed in Non-Patent Document 1.

[0019] In V-DMC, the mesh to be encoded (referred to in this specification as the original mesh) is represented as a base mesh that is less fine (i.e., coarser) than the original mesh, and displacement vectors of the division points obtained by subdividing the base mesh, and the base mesh and displacement vectors are then encoded.

[0020] For example, suppose there is an original mesh as shown in the top row of Figure 2. In Figure 2, black dots represent vertices, and lines connecting the black dots represent connections (edges). As mentioned above, a mesh essentially forms polygons using vertices and edges, but for the sake of convenience, the mesh is described here as a group of vertices connected linearly (in series).

[0021] By decimating some of the vertices of the original mesh, a coarse (low-resolution) mesh is formed as shown in the second row from the top of Figure 2. This is called the base mesh.

[0022] By subdividing each polygon of this base mesh, vertices and edges are added, as shown in the third row from the top of Figure 2. Here, this subdivision adds vertices equal to the number of vertices thinned out from the original mesh. This results in a mesh with the same number of vertices as the original mesh. In this specification, these added vertices are also referred to as division points.

[0023] However, because the connections are updated when the vertices of the original mesh are thinned out, and the division points are formed on these updated connections (edges), the shape of this subdivided base mesh differs from the shape of the original mesh. More specifically, as shown in the bottom part of Figure 2, the positions of the division points (on the dotted lines) differ from those of the original mesh. In this specification, the difference between the positions of the division points and the positions of the vertices of the original mesh is referred to as a displacement vector.

[0024] By representing the original mesh as a base mesh and a displacement vector in this way, the number of polygons (i.e., the number of vertices and edges) is reduced. Therefore, by encoding the base mesh and the displacement vector instead of encoding the original mesh, it is possible to prevent a decrease in encoding efficiency (an increase in the amount of code).

[0025] During decoding, the bitstream is decoded using a decoding method corresponding to the encoding method, and a base mesh and displacement vectors are restored (generated). The base mesh is then subdivided, and the displacement vectors are applied to each division point to restore (generate) the original mesh. In reality, the restored mesh may contain coding artifacts, etc., and may not completely match the original mesh before encoding. However, for the sake of convenience, the following description will be given assuming that the original mesh is restored by decoding, without taking into account coding artifacts, etc. Furthermore, in the following description, polygons are assumed to be triangular.

[0026] <Subdivision of Base Mesh> Next, the subdivision of the base mesh during decoding will be described. As described above, the base mesh has lower resolution (fewer polygons) than the original mesh, so the polygons are divided until the resolution is equivalent to that of the original mesh (i.e., vertices and edges are added to divide the polygons). This subdivision method is arbitrary, but generally, a mesh of the desired resolution is restored (generated) by recursively repeating a process of adding vertices and edges to the base mesh so as to divide each polygon (triangle) into a predetermined number (i.e., hierarchically subdividing the base mesh).

[0027] For example, in a method called midpoint subdivision, a division point (vertex) is added to bisect an edge, thereby dividing the polygon (triangle) into four parts. By repeating this process recursively, a mesh with the desired level of detail is restored (generated). The following description will be given using midpoint subdivision as an example.

[0028] In this subdivision process, three types of information are managed: vertex information, edge information (connection information), and triangle information (polygon information). Vertex information (vertex(float3[])) includes identification information (also called vertex ID) and coordinates of each vertex. Edge information (edges(int2[])) includes identification information (also called edge ID) of each edge and identification information of the vertices located at both ends of that edge. Triangle information (triangles(int3[])) includes identification information (also called triangle ID) of each polygon (triangle) and identification information (vertex ID) of each vertex of that polygon.

[0029] As described above, when subdividing a base mesh, the level of detail is updated hierarchically through recursive processing. That is, vertex information, edge information, and triangle information are also updated hierarchically. In this specification, the level of detail (also referred to as LoD (Level of Detail)) is described assuming that the base mesh (LoD0) is the lowest level, and each time a polygon is divided, the level of detail increases by one level. In other words, the higher the level of detail (i.e., the smaller the polygon), the higher the level of detail (the higher the level of detail).

[0030] For example, let's say the level of resolution of a base mesh is LoD0. In this LoD0, as shown on the left side of Figure 3, the vertices of the base mesh are treated as LoD0 vertices, and information about these vertices is stored in the vertex information. Furthermore, polygons (triangles) formed by connecting these vertices are treated as LoD0 triangles, and information about these triangles is stored in the triangle information. Note that since no edges are formed in LoD0, information about LoD0 edges is not stored in the edge information.

[0031] The mesh of the next higher layer (LoD1) is formed by dividing the polygons of this base mesh once (dividing them into four). For example, as shown in the center of Figure 3, the edges connecting the vertices of LoD0 are treated as edges of LoD1, and information about these edges is stored in the edge information. The midpoints of these LoD1 edges are treated as vertices of LoD1, and information about these vertices is stored (added) in the vertex information. The polygons (triangles) formed by connecting the vertices of LoD0 and LoD1 are treated as triangles of LoD1, and information about these triangles is stored in the triangle information (the triangle information is updated).

[0032] Similarly, a mesh in the next higher layer (LoD2) is formed by dividing (dividing) the polygons of a mesh in LoD1 once (into four). For example, as shown on the right side of Figure 3, the edges connecting the vertices of LoD1 are treated as edges of LoD2, and information about these edges is stored (added) to the edge information. The midpoints of these LoD2 edges are treated as vertices of LoD2, and information about these vertices is stored (added) to the vertex information. The polygons (triangles) formed by connecting the vertices of LoD0 to LoD2 are treated as triangles of LoD2, and information about these triangles is stored (updated) in the triangle information.

[0033] <Updating Edge Information> A more detailed explanation will be given. For example, as shown in FIG. 4, assume that vertices v0 to v4 and triangles t0 and t1 are formed in a base mesh (LoD0). The vertex information (vertex) stores information about each vertex (v0 to v3). In reality, the coordinates of each vertex are also stored, but these are not shown in FIG. 4. The edge information (edges) stores information about edges e0 to e3. However, since edges e0 to e3 do not actually exist, dummy information (-1, -1) is stored instead. Furthermore, the triangle information (triangles) stores information about triangles t0 and t1 (for example, the vertex IDs of the vertices of the triangles ((v0, v1, v2), (v1, v3, v2)), etc.).

[0034] When updating edge information during subdivision, vertexToTriangle and vertexToEdge are generated, and edge information is generated from them. vertexToTriangles is information that links a vertex ID with the triangle ID of the triangle to which the vertex indicated by that vertex ID belongs (the triangle whose vertex is the vertex indicated by the vertex ID). vertexToEdge is information that links a vertex ID with the vertex ID of another vertex connected to that vertex by an edge. Both vertexToTriangle and vertexToEdge must be generated by exhaustively searching the relationship between vertices and triangles.

[0035] First, as shown in Figure 5, each triangle is searched to examine the relationship between the vertices and the triangle, and vertexToTriangles is generated. This information links a vertex to the triangle that touches that vertex (that vertex is a vertex). In other words, vertexToTriangles can derive the triangle ID of the triangle that touches that vertex from its vertex ID. For example, in the case of Figure 5, using vertexToTriangles, t0 is derived from v0. t0 and t1 are derived from v1. t0 and t1 are derived from v2. t1 is derived from v3. In other words, vertexToTriangles indicates that vertex v0 belongs to triangle t0, vertices v1 and v2 belong to triangles t0 and t1, and vertex v3 belongs to triangle t1.

[0036] Next, vertexToEdge is generated as shown in FIG. 6. This information associates each vertex with a nearby vertex connected via an edge. In other words, vertexToEdge makes it possible to derive the vertex ID of another vertex connected to that vertex via an edge from the vertex ID. When generating vertexToEdge, the search for other vertices connected via edges to the target vertex is performed for each triangle using the above-mentioned vertexToTriangles.

[0037] That is, as shown in the example of Figure 6, vertices v1 and v2 are detected as other vertices connected to vertex v0 by edges and belonging to triangle t0. Furthermore, vertices v0 and v2 are detected as other vertices connected to vertex v1 by edges and belonging to triangle t0. Furthermore, vertices v2 and v3 are detected as other vertices connected to vertex v1 by edges and belonging to triangle t1. Furthermore, vertices v0 and v1 are detected as other vertices connected to vertex v2 by edges and belonging to triangle t0. Furthermore, vertices v1 and v3 are detected as other vertices connected to vertex v2 by edges and belonging to triangle t1. Furthermore, vertices v1 and v2 are detected as other vertices connected to vertex v3 by edges and belonging to triangle t1.

[0038] The vertices detected in this way are added to vertexToEdge. However, vertices that have already been detected in other triangles are not added to vertexToEdge to avoid duplication. For example, vertex v2, which belongs to triangle t1 detected as another vertex connected to vertex v1 by an edge, is not added to vertexToEdge because it is also detected as a vertex belonging to triangle t0. Similarly, vertex v1, which belongs to triangle t1 detected as another vertex connected to vertex v2 by an edge, is not added to vertexToEdge because it is also detected as a vertex belonging to triangle t0. This process must be performed sequentially.

[0039] Next, the vertex information is updated using this vertexToEdge. That is, as in the example shown in FIG. 7, the connection between vertices indicated by vertexToEdge is treated as an edge, and that information is added to the edge information. However, information about edges that overlap with edges already stored in the edge information is not stored. That is, overlapping edges are detected. For example, in the case of FIG. 7, by using vertexToEdge, it is possible to find vertex v1 from vertex v0 and to find vertex v0 from vertex v1. That is, vertexToEdge contains information about the edge from vertex v0 to vertex v1 and the edge from vertex v1 to vertex v0. Since these edges are identical (overlapping), they are stored in the edge information as a single edge e4(v0, v1). Similarly, vertexToEdge contains information about the edge from vertex v0 to vertex v2 and the edge from vertex v2 to vertex v0. Since these edges are identical (overlapping), they are stored in the edge information as a single edge e5(v0, v2). Furthermore, vertexToEdge contains information on the edge from vertex v1 to vertex v2 and the edge from vertex v2 to vertex v1. Because these edges are identical (overlapping), the edge information stores them as a single edge e6(v1, v2). Furthermore, vertexToEdge contains information on the edge from vertex v1 to vertex v3 and the edge from vertex v3 to vertex v1. Because these edges are identical (overlapping), the edge information stores them as a single edge e7(v1, v3). Furthermore, vertexToEdge contains information on the edge from vertex v2 to vertex v3 and the edge from vertex v3 to vertex v2. Because these edges are identical (overlapping), the edge information stores them as a single edge e8(v2, v3).

[0040] As described above, while avoiding duplication, edges e4 to e8 are added to the edge information as edges to be processed. That is, vertexToTriangle and vertexToEdge are generated, and edge information can be generated from them.

[0041] <Updating Vertex Information> When vertex information is updated during subdivision, the vertex information is updated using the edge information updated as described above. That is, as shown in FIG. 8, the midpoints of the edges of the LoD to be processed that were generated (added) as described above are added to the vertex information as vertices of the LoD to be processed. That is, in FIG. 8, the midpoint of edge e4 is added as vertex v4. The midpoint of edge e5 is added as vertex v5. The midpoint of edge e6 is added as vertex v6. The midpoint of edge e7 is added as vertex v7. The midpoint of edge e8 is added as vertex v8. In this way, the edge IDs and the vertex IDs of the vertices formed on those edges correspond to each other.

[0042] <Updating Triangle Information> When updating triangle information during subdivision, it is updated using the edge information and vertex information updated as described above. In other words, triangles of the LoD to be processed are generated by dividing the triangle in the next lower layer into four parts, using any three of the vertices generated so far as vertices, and this information is stored in the triangle information. For example, as shown in Figure 9, triangles t0 to t7 are formed, each using any three of vertices v0 to v8 as vertices, and this information is stored in the triangle information.

[0043] To generate a triangle, edgeToVertex is used, which returns the index of the midpoint of the edge ab from the vertex pair a and b. edgeToVertex can be expressed as follows:

[0044] edgeToVertex[a][b]=id edges[id] = (a,b) vertex[id]=midpoint of a and b

[0045] For example, as shown in FIG. LoD0 When [t] = (a, b, c), the indices (a to f) of the vertices that divide the triangle can be calculated using the following three equations.

[0046] edgeToVertex[a][b]=d edgeToVertex[b][c]=e edgeToVertex[a][c]=f

[0047] Therefore, the triangle information can be updated according to the following four equations: In other words, the triangle information can be updated so as to divide the triangle into four parts.

[0048] triangles LoD1 [t]←(d, f, e) triangles LoD1 [u]←(a, d, e) triangles LoD1 [v]←(d, b, f) triangles LoD1 [w]←(e, f, c)

[0049] This edgeToVertex generation involves sequential processing.

[0050] As mentioned above, in the case of midpoint subdivision, when updating triangle information, the triangle in the next lower layer is divided into four (id now :0 to Triangle count - 1). The triangle after division that is in the center of the triangle before division is assigned the same ID as the triangle before division. In other words, the information of the triangle after division that is in the center of the triangle before division is written in the same position in the triangle information as the triangle before division (id center =id now The information for the other three triangles is stored after the information for the central triangle. other = 3 + id now + Triangle count ) and (Id other + 0, id other + 1, id other + 2).

[0051] For example, in the case of Figure 11, triangle t0 on the left side of the figure is divided into four to generate triangles t0, t4, t5, and t6 on the right side of the figure. In this case, information about triangle t0 after division is stored at the position of triangle t0 before division, and information about triangles t4 to t6 is stored after triangles t0 to t3 after division.

[0052] <Increased Processing Time Due to Sequential Processing> As described above, updating edge information and triangle information requires sequential processing, such as full search and duplicate detection, in which processing is executed depending on the results of other processing. This makes it difficult to parallelize the element processing that constitutes the subdivision processing, and it is necessary to execute the processing sequentially. This can increase the processing time for the subdivision.

[0053] 3. Independence of Elemental Processes in Subdivision Processing> Therefore, it is possible to independently execute each elemental process in the subdivision processing. For example, an information processing device may include a decoder that decodes a bitstream and generates a base mesh, a subdivision unit that subdivides the base mesh to a desired level of fineness by recursively repeating a first generation process that generates edges and vertices of a layer of fineness to be processed independently for each edge in a layer one level lower in fineness, a second generation process that generates edges and vertices of a layer of fineness to be processed for a central triangle in the layer one level lower, and a third generation process that generates triangles of a layer of fineness to be processed independently for each triangle in the layer one level lower, up to a desired layer, and a displacement vector application unit that applies displacement vectors to division points, which are vertices of the base mesh subdivided to the desired level of fineness.

[0054] In addition, in an information processing device, a bit stream may be decoded, a base mesh may be generated, and a first generation process may be performed to generate edges and vertices of the layer to be processed of a degree of resolution independently for each edge in the layer one level lower in resolution, a second generation process may be performed to generate edges and vertices of the layer to be processed for the central triangle in the layer one level lower in resolution, and a third generation process may be performed to generate triangles of the layer to be processed independently for each triangle in the layer one level lower, all of which may be repeated recursively up to the desired layer, thereby subdividing the base mesh to the desired degree of resolution and applying a displacement vector to the division points, which are the vertices of the base mesh subdivided to the desired degree of resolution.

[0055] The base mesh is a mesh with lower resolution than the original mesh to be coded, which is composed of vertices and connections that represent the three-dimensional structure of the object, and is generated by thinning out vertices from the original mesh. The displacement vector indicates the difference in position between the division points and the vertices of the original mesh.

[0056] <Decoding device> Fig. 12 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which the present technology is applied. The decoding device 100 shown in Fig. 12 is a device that decodes encoded data of meshes. The decoding device 100 decodes encoded data of meshes that have been encoded using a method basically similar to V-DMC described in Non-Patent Document 1.

[0057] Fig. 12 shows the main processing units, data flows, etc., but does not necessarily show everything. In other words, in the decoding device 100, there may be processing units that are not shown as blocks in Fig. 12, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 12.

[0058] 12 , the decoding device 100 includes a demultiplexing unit 111, a base mesh decoding unit 112, a displacement video decoding unit 113, an atlas data decoding unit 114, an attribute video decoding unit 115, a subdivision unit 116, and a displacement vector application unit 117. Note that the base mesh decoding unit 112, the displacement video decoding unit 113, the atlas data decoding unit 114, and the attribute video decoding unit 115 may be combined into a single processing unit, namely, a decoding unit 121.

[0059] The decoding device 100 receives a bitstream generated by an encoding device that encodes meshes using the V-DMC method. Specifically, this bitstream stores encoded data for a base mesh, encoded data for displacement video, encoded data for atlas data, and encoded data for attribute video. The demultiplexing unit 111 demultiplexes the bitstream and extracts each piece of encoded data contained in the bitstream. For example, the demultiplexing unit 111 extracts the encoded data for the base mesh from the bitstream and supplies it to the base mesh decoding unit 112. The demultiplexing unit 111 also extracts the encoded data for displacement video from the bitstream and supplies it to the displacement video decoding unit 113. The demultiplexing unit 111 also extracts the encoded data for atlas data from the bitstream and supplies it to the atlas data decoding unit 114. The demultiplexing unit 111 also extracts the encoded data for attribute video from the bitstream and supplies it to the attribute video decoding unit 115.

[0060] The base mesh decoding unit 112 decodes the coded data of the base mesh supplied from the demultiplexing unit 111 using a decoding method corresponding to the coding method, and generates (restores) the base mesh. The base mesh decoding unit 112 supplies the generated base mesh to the subdivision unit 116.

[0061] The displacement video decoding unit 113 decodes the encoded data of the displacement video supplied from the demultiplexing unit 111 using a decoding method corresponding to the encoding method, thereby generating (reconstructing) the displacement video. The displacement video is a moving image in which planes onto which displacement vectors are mapped are used as frames. For example, in an encoding device, the displacement vectors are stored in frames at each time point and encoded as the displacement video using a moving image encoding method. The displacement video decoding unit 113 decodes the encoded data using a moving image decoding method corresponding to the moving image encoding method, thereby generating a displacement video that stores the displacement vectors. Note that the displacement video decoding unit 113 may perform the decoding using atlas data supplied from the atlas data decoding unit 114. The displacement video decoding unit 113 supplies the generated displacement video (i.e., displacement vectors) to the displacement vector application unit 117.

[0062] The atlas data decoding unit 114 decodes the coded data of the atlas data supplied from the demultiplexing unit 111 using a decoding method corresponding to the coding method, thereby generating (reconstructing) the atlas data. The atlas data may include various information related to the base mesh and the displacement vector. For example, the atlas data may include information used for subdividing the base mesh. The atlas data may also include information used for decoding the coded data of the displacement video. The atlas data may also include information used for extracting the displacement vector. The atlas data decoding unit 114 supplies the generated atlas data to the displacement video decoding unit 113, the subdivision unit 116, and the displacement vector application unit 117.

[0063] The attribute video decoding unit 115 decodes the coded data of the attribute video supplied from the demultiplexing unit 111 using a decoding method corresponding to the coding method, and generates (restores) the attribute video. The attribute video is a moving image in which planes onto which mesh attributes are mapped are used as frames. The attribute video decoding unit 115 outputs the frame images of the generated attribute video, i.e., the attribute map corresponding to the decoded mesh, to the outside of the decoding device 100. This attribute map is used for rendering, for example, in a subsequent device.

[0064] The subdivision unit 116 subdivides the base mesh supplied from the base mesh decoding unit 112, and supplies the subdivided base mesh to the displacement vector application unit 117. Note that the subdivision unit 116 may subdivide the base mesh using the atlas data supplied from the atlas data decoding unit 114.

[0065] The displacement vector application unit 117 extracts (unpacks) a displacement vector from each frame of the displacement video supplied from the displacement video decoding unit 113. The displacement vector application unit 117 also applies the extracted displacement vector to the vertices (division points) of the subdivided base mesh supplied from the subdivision unit 116 to reconstruct a mesh. In this specification, this reconstructed mesh is also referred to as a decoded mesh. In other words, it can also be said that the displacement vector application unit 117 generates a decoded mesh. The displacement vector application unit 117 may also apply the displacement vector using the atlas data supplied from the atlas data decoding unit 114. The displacement vector application unit 117 outputs the generated decoded mesh to the outside of the decoding device 100. This decoded mesh, together with the attribute map described above, is used for rendering, for example, in a subsequent device.

[0066] In the decoding device 100 configured as above, the subdivision unit 116 independently executes each element process of the subdivision process, as described above. For example, as shown in FIG. 13 , the subdivision unit 116 performs the update of vertices and edges by dividing it into the following two generation processes:

[0067] In the first generation process, the subdivision unit 116 generates vertices and edges of the current LoD located on each edge of the next lower LoD (updating on an edge-by-edge basis). That is, the subdivision unit 116 divides each edge of the next lower LoD in half, sets each half as an edge of the current LoD, and sets the midpoint of each edge of the current LoD as a vertex of the current LoD. These vertices and edges are generated independently for each edge of the next lower LoD. In other words, the subdivision unit 116 can generate vertices and edges on the current edge of the next lower LoD independently of the process of generating vertices and edges on other edges of the next lower LoD or the results of that process. Therefore, the subdivision unit 116 generates these vertices and edges independently for each edge of the next lower LoD.

[0068] In the second generation process, the subdivision unit 116 generates vertices and edges of the current LoD located on each side of the central triangle of the current LoD (updating on a triangle-by-triangle basis). That is, the subdivision unit 116 defines each side of the central triangle as an edge of the current LoD, and defines the midpoints of each edge of the current LoD as vertices of the current LoD. These vertices and edges are generated independently for each triangle. In other words, the subdivision unit 116 can generate vertices and edges located on each side of the current triangle independently of the processes for generating vertices and edges located on the sides of other triangles or the results of those processes. Therefore, the subdivision unit 116 generates these vertices and edges independently for each central triangle of the current LoD.

[0069] The subdivision unit 116 also executes a third generation process to generate triangles of the target LoD independently for each triangle of the next lower LoD, using the vertices and edges generated by the first generation process and the second generation process.

[0070] In this way, the subdivision unit 116 can independently execute each element process of the subdivision process. For example, the subdivision unit 116 can independently execute element processes for each edge of the next lower LoD and element processes for each triangle. Therefore, for example, the subdivision unit 116 can execute multiple element processes in parallel. Therefore, the subdivision unit 116 can suppress an increase in the processing time of the subdivision process. Therefore, the decoding device 100 can suppress an increase in the processing time for decoding meshes.

[0071] <Flow of Decoding Process> An example of the flow of the decoding process executed by the decoding device 100 will be described with reference to the flowchart of FIG.

[0072] When the decoding process starts, in step S101, the demultiplexer 111 demultiplexes the bit stream and extracts various types of coded data.

[0073] In step S102, the atlas data decoding unit 114 decodes the coded data of the atlas data extracted in step S101 to generate (restore) the atlas data.

[0074] In step S103, the displaced video decoding unit 113 decodes the encoded data of the displaced video extracted in step S101 to generate (restore) the displaced video. In this case, the displaced video decoding unit 113 may use the atlas data generated in step S102.

[0075] In step S104, the base mesh decoding unit 112 decodes the coded data of the base mesh extracted in step S101, and generates (restores) the base mesh.

[0076] In step S105, the subdivision unit 116 executes a subdivision process to subdivide the base mesh generated in step S104. At this time, the subdivision unit 116 may use the atlas data generated in step S102.

[0077] In step S106, the displacement vector application unit 117 extracts (unpacks) displacement vectors from the displacement video generated in step S103. In this process, the displacement vector application unit 117 may use the atlas data generated in step S102. The displacement vector application unit 117 also applies the extracted displacement vectors to the vertices (division points) of the base mesh subdivided by the subdivision process in step S105, thereby generating (reconstructing) a decoded mesh. The displacement vector application unit 117 outputs the generated decoded mesh to the outside of the decoding device 100.

[0078] In step S107, the attribute video decoding unit 115 decodes the coded data of the attribute video extracted in step S101 to generate an attribute video. The attribute video decoding unit 115 outputs frame images (attribute map) of the attribute video to the outside of the decoding device 100.

[0079] The decoded mesh and attribute map outputted to the outside of the decoding device 100 are used for rendering or the like in another device at a later stage, for example.

[0080] <Flow of Subdivision Processing> Next, an example of the flow of the subdivision processing executed in step S105 of Fig. 14 will be described with reference to the flowchart of Fig. 15. The description will also refer to Figs. 16 to 18 as necessary.

[0081] When the subdivision process starts, in step S131, the subdivision unit 116 generates edge information of the LoD1 based on the base mesh and updates the triangle information. In step S132, the subdivision unit 116 also generates vertex information of the LoD1 based on the base mesh. The processes of steps S131 and S132 are also referred to as pre-processing. Any method can be used for this pre-processing. For example, the methods described with reference to FIGS. 3 to 11 may be used.

[0082] In other words, the subdivision unit 116 may generate edges, vertices, and triangles of the second layer of resolution based on a base mesh, which is the lowest layer of resolution, and then perform the first to third generation processes described below for the third and subsequent layers of resolution.

[0083] In step S133, the subdivision unit 116 updates the processing target LoD to the next higher layer.

[0084] In step S134, the subdivision unit 116 generates edge information and vertex information of the current LoD for each edge of the next lower LoD (first generation process). That is, in the first generation process, the subdivision unit 116 generates vertices and edges of the current LoD located on each edge of the next lower LoD (updating on an edge-by-edge basis). For example, the subdivision unit 116 divides each edge of the next lower LoD into two equal parts, sets each half as an edge of the current LoD, and sets the midpoint of each edge of the current LoD as a vertex of the current LoD. The subdivision unit 116 generates these vertices and edges independently for each edge of the next lower LoD.

[0085] For example, suppose the subdivision unit 116 performs the first generation process as shown in Fig. 16 to generate vertices and edges of LoD2 located on each edge of LoD1. For example, when the first generation process is performed on edge (0, 1) of index 3 of LoD1, which corresponds to vertex of index 3 of LoD1, edges (0, 3) of index 6 and edges (1, 3) of index 7 of LoD2 are generated. In addition, a vertex of index 6, which is the midpoint of the edge of index 6, and a vertex of index 7, which is the midpoint of the edge of index 7, are generated. These vertices are vertices of LoD2.

[0086] Edge Count indicates the number of edges at the next lower LoD, and Edge Offset indicates the offset value of the edge information storage position, and Triangle Count If Edge denotes the number of triangles in the LoD to be processed, then in the example of FIG. 16, these values ​​are as follows: Count = 3 Edge Offset= 3 Triangle Count = 4

[0087] In the first generation process, the existing edge index id edge Two edges are added to each edge. id_edge: 0 to (Edge Count - 1) id' = 2 * id edge + Edge Offset + Edge count

[0088] The indexes to be written are id' + 0, id' + 1. Therefore, the update of edge information can be expressed as follows: edges[id' + 0] = (edges[id edge ][0], id edges ) edges[id′ + 1] = (edges[id edge ][1], id edges )

[0089] In step S135, the subdivision unit 116 generates edge information and vertex information of the LoD to be processed for the central triangle (second generation process). That is, in the second generation process, the subdivision unit 116 generates vertices and edges of the LoD to be processed that are located on each side of the central triangle of the LoD to be processed (updating in units of triangles). For example, the subdivision unit 116 sets each side of the central triangle as an edge of the LoD to be processed, and sets the midpoints of each edge of the LoD to be processed as vertices of the LoD to be processed. The subdivision unit 116 generates these vertices and edges independently for each central triangle of the LoD to be processed.

[0090] For example, suppose the subdivision unit 116 performs the second generation process as shown in Fig. 17 to generate vertices and edges of LoD2 located on each side of the central triangle of LoD1. In this case, the sides (3, 5), (4, 5), and (3, 4) of the central triangle 0 (3, 5, 4) of LoD1, shown in gray, are set as edges of LoD2 (indexes 12 to 14), and the midpoints of these edges are set as vertices of LoD2 (indexes 12 to 14).

[0091] In this case, Edge Count , Edge Offset , and Triangle Count The value of Edge is as follows: Count = 3 Edge Offset = 3 Triangle Count = 4

[0092] Then, in the second generation process, the existing triangle index id tri Three sides are added to each id. tri : 0 to (Triangle Count / 4 - 1) id' = 3 * id tri + Edge Offset + 3 * Edge count

[0093] The indices to be written are id' + 0, id' + 1, id' + 2. Therefore, the update of edge information can be expressed as follows: edges[id' + 0] = (triangles[0], triangles[1]) edges[id' + 1] = (triangles[1], triangles[2]) edges[id' + 2] = (triangles[2], triangles[0])

[0094] In step S136, the subdivision unit 116 updates the triangle information (third generation process). That is, in the third generation process, the subdivision unit 116 generates triangles of the current LoD independently for each triangle of the next lower LoD using the vertices and edges generated by the first and second generation processes. The subdivision unit 116 generates triangles of the current LoD independently for each triangle.

[0095] The third generation process may be performed in any manner. For example, information that can be handled in the same way as egdeToVertex may be prepared before the recursive subdivision process is repeated, and the information may be updated when the recursive subdivision process is repeated.

[0096] For example, in the third generation process, the subdivision unit 116 may update edgeToVertex, which indicates other vertices linked to a desired vertex via an edge, using edges and vertices in the layer to be processed, and generate triangles in the layer to be processed using the updated edgeToVertex. For example, the subdivision unit 116 may generate edgeToVertex of LoD1 in pre-processing, and update edgeToVertex in the same way as the edge information when recursively repeating the process.

[0097] The triangle information may be updated in the same manner as in the conventional method. In this case, the subdivision unit 116 further updates edgeToVertex. The subdivision unit 116 updates edgeToVertex together with updating the edge information and vertex information.

[0098] For example, in the case of FIG. 18, the subdivision unit 116 may add a vertex g and an edge g as follows: edges_LoD2[g]←(a, d) edgeToVertex[a][d]←g

[0099] By doing so, the subdivision unit 116 can apply a method for updating triangle information in a conventional method.

[0100] In addition, in the third generation process, the subdivision unit 116 may derive edges in the target layer corresponding to each side of the triangle in the next lower layer, and generate a triangle in the target layer using the derived edges. For example, the subdivision unit 116 may construct triangle information in advance so that three edges, rather than three vertices, are obtained from a triangle, and derive the information from the edge list.

[0101] Since the triangle information indicates edge information, the subdivision unit 116 can obtain the vertex indices a to f of the triangles after division from this information as follows: LoD0 [t] = (d, f, e) edges[d] = (a, b) edges[f] = (b, c) edges[e] = (a, c)

[0102] The subdivision unit 116 converts this into edges. Therefore, the indices of the edges of (d, f), (e, f), and (d, e) can be derived as follows: edges[3t + 3edge Count + edge Offset ] = (d, f) edges[3t + 3edge Count + edge Offset + 1] = (e, f) edges[3t + 3edge Count + edge Offset + 2] = (d, e)

[0103] In addition, the indices of edges (a, d), (b, d), etc. can be derived as follows: edges[2d + edge Count + edge Offset ] = (a, d) edges[2d + edge Count + edge Offset + 1] = (b, d)

[0104] This allows the indexes of all edges that make up the triangle after division to be found, making it possible to update the triangle.

[0105] In this way, the subdivision unit 116 can suppress an increase in required resources because the data generated in advance is only the edges of LoD1. Also, the subdivision unit 116 can suppress an increase in the load of data transfer.

[0106] Alternatively, triangle information of LoD1 may be generated in advance. For example, in the third generation process, the subdivision unit 116 may generate the remaining triangles in the target layer and the central triangle in the next higher layer of resolution based on the triangles in the next lower layer and the central triangle in the target layer.

[0107] 0 ≦ t < triangleCount LoD0 In triangles LoD0 [t] = (a, b, c), triangles LoD1When [t] = (d, f, e), edges[d], edges[f], and edges[e] can be calculated from the triangle update rules of the conventional method. Therefore, triangles can be updated in the same way as the method described above. For example, at the iteration of the i-th iteration, the triangle of LoD I + 1 is updated. edges[d] = (a, b) edges[f] = (b, c) edges[e] = (a, c)

[0108] By doing so, edgeToVertex is not required, and therefore an increase in required resources can be suppressed compared to the above-mentioned method. Also, the subdivision unit 116 can suppress an increase in the load of data transfer.

[0109] In step S137, the subdivision unit 116 determines whether or not subdivision has been performed to the desired LoD, that is, whether or not subdivision has been performed to the same level of fineness as the original mesh.

[0110] If it is determined that the base mesh has not been subdivided to the desired LoD, i.e., if it is determined that the level of definition is still lower (coarser) than that of the original mesh, the subdivision unit 116 returns to step S133 and repeats the subsequent steps. That is, the processes of steps S133 to S137 are repeated recursively until the base mesh is subdivided to the desired LoD.

[0111] Then, in step S137, if it is determined that subdivision has been performed to the desired LoD (subdivision has been performed to the same level of fineness as the original mesh), the subdivision process ends and the process returns to FIG.

[0112] By performing the subdivision process as described above, the subdivision unit 116 can independently perform each element process of the subdivision process. For example, the subdivision unit 116 can independently perform element processes for each edge of the next lower LoD and element processes for each triangle. Therefore, for example, the subdivision unit 116 can execute multiple element processes in parallel. Therefore, the subdivision unit 116 can suppress an increase in the processing time of the subdivision process. Therefore, the decoding device 100 can suppress an increase in the processing time for decoding meshes.

[0113] <Parallel Processing> As described above, the subdivision unit 116 can independently execute each element process of the subdivision process. Therefore, the subdivision unit 116 can execute each element process in parallel, for example, as shown in graph 202 in FIG.

[0114] In Fig. 19, each square represents an element process (thread). As shown in Fig. 19, subdivision is performed for each level of detail (LoD). In the processing of each level, as shown in graph 201, conventional methods do not allow each element process to be executed in parallel. This could result in an increase in processing time.

[0115] In contrast to this, in the case of the above-described technology, element processes can be executed independently of each other in each layer, and therefore, multiple element processes can be executed in parallel, as shown in graph 202 in Fig. 19 .

[0116] For example, in graph 202, the diagonally shaded squares in the upper left and lower right indicate threads of the first generation process, and the numbers indicate the indexes of each thread. As shown in graph 202, in the processing of each LoD, the subdivision unit 116 may execute multiple threads of this first process in parallel. In other words, the subdivision unit 116 may execute the first generation processes, which are performed independently for each edge of the next lower layer, in parallel.

[0117] In addition, hatched boxes indicate threads of the second generation process, and numbers indicate the index of each thread. As shown in graph 202, in the process of each LoD, the subdivision unit 116 may execute multiple threads of this second process in parallel with each other.

[0118] Furthermore, as shown in the graph 202, in the processing of each LoD, the subdivision unit 116 may execute the first processing and the second processing in parallel.

[0119] The diagonally shaded squares in the upper right and lower left indicate threads of the third generation process, and the numbers indicate the indexes of each thread. As shown in graph 202, in the processing of each LoD, the subdivision unit 116 may execute multiple threads of this third process in parallel. In other words, the subdivision unit 116 may execute the third generation process, which is performed independently for each triangle in the next lower layer, in parallel.

[0120] The subdivision unit 116 may also execute the first to third generation processes in parallel for each triangle in the two lowest layers of resolution.

[0121] By performing multiple processes in parallel in this way, the subdivision unit 116 can suppress an increase in the processing time for the subdivision process, and therefore the decoding device 100 can suppress an increase in the processing time for decoding the meshes.

[0122] <CPU, GPU> The above-described segmentation process (each step in FIG. 15 ) may be executed by any hardware. For example, it may be executed by a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). Generally, a GPU has higher parallel processing performance than a CPU, and can process the segmentation process to which the present technology is applied at a higher speed. In other words, an increase in processing time can be suppressed. Furthermore, by using a GPU, the CPU resource occupancy rate can be reduced, and therefore an increase in processing time for processes other than the segmentation process executed by the CPU can be suppressed.

[0123] <Others> Although the above description uses midpoint subdivision as an example of a subdivision method, any subdivision method can be used. For example, vertices may be formed at positions other than the midpoints of edges. Furthermore, the number of vertices, edges, and triangles added by one subdivision (one LoD processing) may be other than the above example.

[0124] In the above description, the polygons are assumed to have a triangular shape, but this shape is merely an example and the polygons may have any other polygonal shape.

[0125] <4. Supplementary Note> <Control of Application of the Present Technology> Note that another device such as an encoder may control whether a decoder (e.g., the decoding device 100) applies the present technology. For example, flag information indicating whether to apply the present technology to decode a mesh may be associated with a bitstream as metadata and supplied to the decoder. In this case, the decoder may determine whether to apply the present technology to decode a mesh based on the flag.

[0126] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0127] FIG. 20 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0128] In a computer 900 shown in FIG. 20, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0129] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0130] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 912 includes, for example, a display, a speaker, and an output terminal. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 914 includes, for example, a network interface. The drive 915 drives a removable recording medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0131] In a computer configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0132] The program executed by the computer can be applied by recording it on a removable recording medium 921 such as a package medium, for example. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by attaching the removable recording medium 921 to the drive 915.

[0133] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0134] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0135] <Application of the Present Technology> The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0136] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set in which other functions are added to a unit (e.g., a video set).

[0137] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0138] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0139] <Fields and uses to which this technology can be applied> Systems, devices, processing units, etc. to which this technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, the uses thereof are also arbitrary.

[0140] <Others> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in a bitstream, but also in a bitstream that includes differential information of the identification information relative to certain reference information. Therefore, in this specification, "flag" and "identification information" encompass not only the information itself, but also differential information relative to the reference information.

[0141] Furthermore, various types of information (e.g., metadata) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, the associated pieces of data may be combined into one piece of data or may be separate pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Furthermore, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer not to the entire data, but to only part of the data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0142] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0143] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0144] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0145] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0146] Also, for example, each step of a single flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, multiple processes included in a single step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0147] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0148] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0149] Note that the present technology can also be configured as follows: (1) An information processing device comprising: a decoding unit that decodes a bitstream and generates a base mesh; a subdivision unit that subdivides the base mesh to a desired level of fineness by recursively repeating, up to a desired level, a first generation process that generates edges and vertices of a layer of fineness to be processed independently for each edge in the layer one level lower, a second generation process that generates edges and vertices of the layer of fineness to be processed for a central triangle in the layer one level lower, and a third generation process that generates triangles of the layer of fineness to be processed independently for each triangle in the layer one level lower; and a displacement vector application unit that applies displacement vectors to division points that are vertices of the base mesh subdivided to the desired level of fineness, wherein the base mesh is a mesh of lower resolution than an original mesh to be coded that is composed of vertices and connections that represent a three-dimensional structure of an object, and the displacement vectors indicate differences in position between the division points and the vertices of the original mesh. (2) The information processing device according to (1), wherein the subdivision unit executes the first generation processes, which are performed independently for each edge in the layer one level lower, in parallel with each other. (3) The information processing device according to (1) or (2), wherein the subdivision unit executes the third generation processes, which are performed independently for each triangle in the layer one level lower, in parallel with each other. (4) The information processing device according to any of (1) to (3), wherein the subdivision unit executes the first generation process and the second generation process in parallel with each other. (5) The information processing device according to any of (1) to (4), wherein the subdivision unit executes the first generation process to the third generation process in parallel for each triangle in the layer two levels lower in resolution. (6) The information processing device according to any of (1) to (5), wherein the subdivision unit, in the third generation process, updates edgeToVertex, which indicates another vertex linked to a desired vertex via an edge, using edges and vertices in the layer to be processed.(7) The information processing device according to any one of (1) to (5), wherein the subdivision unit, in the third generation process, derives edges of the processing target layer corresponding to each side of the triangle in the next lower layer, and generates triangles of the processing target layer using the derived edges. (8) The information processing device according to any one of (1) to (5), wherein the subdivision unit, in the third generation process, generates remaining triangles of the processing target layer and a central triangle of the next higher layer of resolution based on the triangle in the next lower layer and a central triangle of the processing target layer. (9) The information processing device according to any one of (1) to (8), wherein the subdivision unit generates edges, vertices, and triangles of the second layer of resolution based on the base mesh that is the lowest layer of resolution, and executes the first generation process to the third generation process for the third and subsequent layers of resolution. (10) An information processing method comprising: decoding a bitstream; generating a base mesh; recursively repeating, up to a desired layer, a first generation process for generating edges and vertices of a layer of resolution to be processed independently for each edge in the layer one level lower; a second generation process for generating edges and vertices of the layer of resolution to be processed for a central triangle in the layer one level lower; and a third generation process for generating triangles of the layer of resolution to be processed independently for each triangle in the layer one level lower; thereby subdividing the base mesh to a desired level of resolution; applying displacement vectors to division points, which are vertices of the base mesh subdivided to the desired level of resolution; the base mesh is a mesh of lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; and the displacement vector indicates a difference in position between the division points and the vertices of the original mesh.

[0150] 100 Decoding device, 111 Demultiplexing unit, 112 Base mesh decoding unit, 113 Displacement video decoding unit, 114 Atlas data decoding unit, 115 Attribute video decoding unit, 116 Subdivision unit, 117 Displacement vector application unit, 121 Decoding unit, 900 Computer

Claims

1. An information processing device comprising: a decoding unit that decodes a bit stream and generates a base mesh; a subdivision unit that subdivides the base mesh to a desired level of fineness by recursively repeating, up to a desired level, a first generation process that generates edges and vertices of a hierarchy to be processed of a level of fineness independently for each edge of a layer one level lower in fineness, a second generation process that generates edges and vertices of the hierarchy to be processed for a central triangle in the layer one level lower in fineness, and a third generation process that generates triangles of the hierarchy to be processed independently for each triangle in the layer one level lower in fineness; and a displacement vector application unit that applies displacement vectors to division points, which are vertices of the base mesh subdivided to the desired level of fineness, wherein the base mesh is a mesh of lower fineness than an original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and the displacement vector indicates a difference in position between the division points and the vertices of the original mesh.

2. The information processing device according to claim 1, wherein the subdivision unit executes the first generation processes, which are performed independently for each edge in the next lower layer, in parallel with each other.

3. The information processing device according to claim 1, wherein the subdivision unit executes the third generation processes, which are performed independently for each triangle in the next lower layer, in parallel with each other.

4. The information processing device according to claim 1, wherein the subdivision unit executes the first generation process and the second generation process in parallel with each other.

5. The information processing device according to claim 1, wherein the subdivision unit executes the first generation process through the third generation process in parallel for each triangle in the layer two levels lower than the resolution.

6. The information processing device according to claim 1, wherein in the third generation process, the subdivision unit updates edgeToVertex, which indicates other vertices linked to a desired vertex via an edge, using edges and vertices of the hierarchy to be processed, and generates a triangle of the hierarchy to be processed using the updated edgeToVertex.

7. The information processing device according to claim 1, wherein the subdivision unit, in the third generation process, derives edges of the hierarchy to be processed that correspond to each side of the triangle in the next lower hierarchy, and generates a triangle in the hierarchy to be processed using the derived edges.

8. The information processing device according to claim 1, wherein the subdivision unit, in the third generation process, generates the remaining triangles of the hierarchy to be processed and a central triangle of the hierarchy one level higher in resolution based on the triangle of the next lower layer and the central triangle of the hierarchy to be processed.

9. The information processing device according to claim 1, wherein the subdivision unit generates edges, vertices, and triangles of the second level of resolution based on the base mesh, which is the lowest level of resolution, and performs the first generation process through the third generation process for the third and subsequent levels of resolution.

10. An information processing method comprising: decoding a bit stream to generate a base mesh; recursively repeating, up to a desired hierarchy, a first generation process for generating edges and vertices of a hierarchy to be processed of a level of resolution independently for each edge of a layer one level lower in resolution; a second generation process for generating edges and vertices of the hierarchy to be processed for a central triangle in the layer one level lower in resolution; and a third generation process for generating triangles of the hierarchy to be processed independently for each triangle in the layer one level lower in resolution, thereby subdividing the base mesh to a desired level of resolution; applying a displacement vector to division points, which are vertices of the base mesh subdivided to the desired level of resolution; the base mesh is a mesh of lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; and the displacement vector indicates a difference in position between the division points and the vertices of the original mesh.