Information processing device and method

Abstract

The present disclosure relates to an information processing device and method with which it is possible to suppress an increase in the load of decoding processing of 3D data including a base mesh and a displacement vector. Edge preservation information is generated, atlas information including the edge preservation information is encoded, and a bitstream including encoded data of a base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information is generated. The bitstream is demultiplexed, the encoded data of the atlas information is decoded, the encoded data of the base mesh is decoded, an initial edge list of the base mesh is generated by using the edge preservation information included in the atlas information, and the base mesh is subdivided by using the initial edge list. The present disclosure can be applied to, for example, information processing devices, electronic equipment, information processing methods, and programs.

Description

Information processing device and method

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that can suppress an increase in the load of decoding 3D data including a base mesh and a displacement vector.

[0002] Conventionally, V-DMC (Video-based Dynamic Mesh Coding) has been used as a method for encoding meshes, which are 3D data that represent the three-dimensional structure of an object using vertices and connections (see, for example, Non-Patent Document 1). In the decoding process of this V-DMC, an initial edge list indicating the edges of the base mesh is derived before the base mesh is subdivided. The edges to be included in this initial edge list are derived from combinations of adjacent vertices of the base mesh.

[0003] Khaled Mammou, Jungsun Kim, Alexis Tourapis, Dimitri Podborski, Krasimir Kolarov, "[V-CG] Apple's Dynamic Mesh Coding CfP Response", ISO / IEC JTC 1 / SC 29 / WG 7 m59281, April 2022

[0004] However, in deriving this initial edge list, overlap determination is performed for each combination to prevent overlapping edges from being included in the initial edge list, which may increase the load of the decoding process.

[0005] The present disclosure has been made in view of such circumstances, and makes it possible to suppress an increase in the load of decoding 3D data including base meshes and displacement vectors.

[0006] An information processing device according to one aspect of the present technology includes: an edge preservation information generation unit that generates edge preservation information indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list; an atlas information encoding unit that encodes atlas information including the edge preservation information and generates encoded data of the atlas information; and a bitstream generation unit that generates a bitstream including encoded data of the base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information, wherein the base mesh is generated by thinning out vertices from an original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object. the displacement video is a moving image having frame images of a displacement map, which is a two-dimensional area into which displacement vectors indicating the displacement of the vertices of the subdivided base mesh are packed; the attribute video is a moving image having frame images of an attribute map, which is a two-dimensional area into which the texture of the original mesh is packed; the atlas information is information used to reconstruct the mesh, including information indicating the correspondence between the base mesh, the displacement map, and the attribute map; and the initial edge list is a unique edge list indicating the edges of the base mesh by the non-overlapping combinations.

[0007] An information processing method according to one aspect of the present technology includes generating edge preservation information indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list, encoding atlas information including the edge preservation information, generating encoded data of the atlas information, and generating a bitstream including encoded data of the base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information, wherein the base mesh is a mesh with lower resolution than the original mesh, which is generated by thinning vertices from an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object. an information processing method in which the displacement video is a moving image having frame images of a displacement map, which is a two-dimensional area into which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; the attribute video is a moving image having frame images of an attribute map, which is a two-dimensional area into which the texture of the original mesh is packed; the atlas information is information used for mesh reconstruction, including information indicating the correspondence between the base mesh, the displacement map, and the attribute map; and the initial edge list is a unique edge list indicating the edges of the base mesh by the non-overlapping combinations.

[0008] An information processing device according to another aspect of the present technology includes: a demultiplexing unit that demultiplexes a bitstream and extracts coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information; an atlas information decoding unit that decodes the coded data of the extracted atlas information; a base mesh decoding unit that decodes the coded data of the extracted base mesh; an initial edge list generating unit that generates an initial edge list of the base mesh using edge preservation information included in the atlas information that indicates whether each combination of adjacent vertices of the base mesh should be included in an initial edge list as an edge; and a subdivision unit that subdivides the base mesh using the initial edge list, wherein the base mesh is subdivided into vertices and connections that represent a three-dimensional structure of an object. the displacement video is a moving image having frame images of a displacement map, which is a two-dimensional area into which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; the attribute video is a moving image having frame images of an attribute map, which is a two-dimensional area into which the texture of the original mesh is packed; the atlas information is information used for mesh reconstruction, including information indicating the correspondence between the base mesh, the displacement map, and the attribute map; and the initial edge list is a unique edge list indicating the edges of the base mesh by the non-overlapping combinations.

[0009] An information processing method according to another aspect of the present technology includes demultiplexing a bitstream, extracting coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information, decoding the extracted coded data of the atlas information, decoding the extracted coded data of the base mesh, generating the initial edge list of the base mesh using edge preservation information included in the atlas information that indicates whether each combination of adjacent vertices of the base mesh should be included in an initial edge list as an edge, and subdividing the base mesh using the initial edge list, and the base mesh is generated by subdividing vertices from an original mesh to be encoded that is configured of vertices and connections that represent a three-dimensional structure of an object. the displacement video is a moving image having frame images of a displacement map, which is a two-dimensional area into which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; the attribute video is a moving image having frame images of an attribute map, which is a two-dimensional area into which the texture of the original mesh is packed; the atlas information is information used for mesh reconstruction, including information indicating the correspondence between the base mesh, the displacement map, and the attribute map; and the initial edge list is a unique edge list indicating the edges of the base mesh by the non-overlapping combinations.

[0010] In an information processing device and method according to one aspect of the present technology, edge preservation information is generated indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list, atlas information including the edge preservation information is encoded, encoded data of the atlas information is generated, and a bitstream including the encoded data of the base mesh, encoded data of the displacement video, encoded data of the attribute video, and encoded data of the atlas information is generated.

[0011] In another aspect of the information processing device and method of the present technology, the bitstream is demultiplexed, and coded data of the base mesh, coded data of the displacement video, coded data of the attribute video, and coded data of the atlas information are extracted, the extracted coded data of the atlas information is decoded, the extracted coded data of the base mesh is decoded, and edge preservation information included in the atlas information indicating whether each combination of adjacent vertices of the base mesh should be included as an edge in the initial edge list is used to generate an initial edge list of the base mesh, and the base mesh is subdivided using the initial edge list.

[0012] FIG. 1 is a diagram for explaining a mesh. FIG. 1 is a diagram for explaining V-DMC. FIG. 1 is a diagram for explaining an outline of a decoding process. FIG. 2 is a diagram for explaining an example of a base mesh. FIG. 3 is a diagram for explaining an example of how an initial edge list is derivated. FIG. 4 is a diagram for explaining an example of a pseudo program for the process of deriving an initial edge list. FIG. 5 is a diagram for explaining an example of how ComputeNeighbours processing is done. FIG. 6 is a diagram for explaining an example of how ComputeNeighbours processing is done. FIG. 7 is a diagram for explaining an outline of a decoding process. A block diagram showing an example of the main configuration of an encoding device. A block diagram showing an example of the main configuration of a V-DMC encoding unit. A flowchart for explaining an example of the flow of an encoding process. A flowchart for explaining an example of the flow of a V-DMC encoding process. A block diagram showing an example of the main configuration of a decoding device. A flowchart for explaining an example of the flow of a decoding process. A diagram for explaining an example of an encoding / decoding method. A diagram for explaining an outline of the decoding process. A diagram for explaining an example of the ComputeNeighbours processing. FIG. 1 is a diagram illustrating an example of ComputeNeighbors processing. FIG. 2 is a block diagram illustrating an example of the main configuration of an encoding device. FIG. 3 is a flowchart illustrating an example of the flow of encoding processing. FIG. 4 is a block diagram illustrating an example of the main configuration of a decoding device. FIG. 5 is a flowchart illustrating an example of the flow of decoding processing. FIG. 6 is a flowchart illustrating an example of the flow of initial edge list generation processing. FIG. 7 is a diagram for explaining an overview of decoding processing. FIG. 8 is a diagram illustrating an example of the flow of initial edge list generation. FIG. 9 is a diagram illustrating an example of the flow of initial edge list generation. FIG. 10 is a flowchart illustrating an example of the flow of initial edge list generation processing. FIG. 11 is a diagram illustrating an example of a base mesh. FIG. 12 is a block diagram illustrating an example of the main configuration of an encoding device. FIG. 13 is a flowchart illustrating an example of the flow of encoding processing. FIG. 14 is a flowchart illustrating an example of the flow of decoding processing.FIG. 1 is a diagram showing an example of a pseudo program for processing to derive an initial edge list. FIG. 2 is a diagram showing an example of a pseudo program for processing to derive an initial edge list. FIG. 3 is a diagram showing an example of a pseudo program for processing to derive an initial edge list. FIG. 4 is a diagram showing an example of a pseudo program for processing to derive an initial edge list. FIG. 5 is a block diagram showing an example of the main configuration of a computer.

[0013] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be made in the following order: 1. Literature supporting technical content and technical terminology 2. Generation of initial edge list in V-DMC decoding 3. Transmission of initial edge list 4. Transmission of edge preservation information 5. Supplementary notes

[0014] <1. Literature, etc. supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the content described in the embodiments, but also the content described in the following non-patent documents, etc. that were publicly known at the time of filing, and the content of other documents referenced in the following non-patent documents.

[0015] Non-patent document 1: (mentioned above)

[0016] In other words, the contents of the above-mentioned non-patent documents and the contents of other documents referenced in the above-mentioned non-patent documents are also used as the basis for determining the support requirements.

[0017] <2. Generating an initial edge list in V-DMC decoding> <V-DMC> Conventionally, 3D data representing the three-dimensional structure of a three-dimensional structure (an object with a three-dimensional shape) has been available as a mesh, which represents the three-dimensional shape of the object surface by forming polygons using vertices and connections (also called edges).

[0018] As shown in the upper left of Figure 1, in a mesh, vertices 11 and connections 12 connecting these vertices 11 form polygonal planes (polygons). These polygons (also called faces) represent the surface of a three-dimensional object, i.e., the three-dimensional shape of the object. A texture 13 can be applied to each face of this mesh.

[0019] Mesh data is composed of information such as that shown in the lower part of Figure 1. Vertex information 14, shown first from the left in the lower part of Figure 1, is information indicating the three-dimensional position (three-dimensional coordinates (X, Y, Z)) of each vertex 11 that constitutes the mesh. Connection information 15, shown second from the left in the lower part of Figure 1, is information indicating each connection (edge) 12 that constitutes the mesh. A texture image 16, shown third from the left in the lower part of Figure 1, is map information for the texture 13 that is applied to each face. A UV map 17, shown fourth from the left in the lower part of Figure 1, is information indicating the correspondence between the vertices 11 and the texture 13. The UV map 17 indicates the coordinates (UV coordinates) of each vertex 11 in the texture image 16.

[0020] As an example of such a mesh coding method, there is V-DMC (Video-based Dynamic Mesh Coding) as disclosed in Non-Patent Document 1.

[0021] In V-DMC, the mesh to be encoded (referred to in this specification as the original mesh) is represented as a base mesh that is less fine (i.e., coarser) than the original mesh, and displacement vectors of the division points obtained by subdividing the base mesh, and the base mesh and displacement vectors are then encoded.

[0022] For example, assume that there is an original mesh as shown in the top row of Figure 2. The original mesh is a mesh composed of vertices and connections that represent the three-dimensional structure of an object, and is the target of encoding. For example, the original mesh is generated from a captured image of an object in real space (by camera capture). In Figure 2, black dots represent vertices, and lines connecting the black dots represent connections (edges). As described above, a mesh essentially forms polygons using vertices and edges, but for convenience of explanation, it is described here as a group of vertices connected linearly (in series).

[0023] By simplifying the original mesh, a coarse (low-resolution) mesh like the one shown in the second row from the top of Figure 2 is formed. This is called the base mesh. One simplification method is to thin out some of the vertices (decimate). In other words, the base mesh is a mesh with lower resolution than the original mesh, generated by thinning out vertices from the original mesh (i.e., simplifying the original mesh).

[0024] By subdividing each polygon of this base mesh, vertices and edges are added, as shown in the third row from the top of Figure 2. The degree of subdivision is arbitrary. That is, the number of vertices and edges added is arbitrary. For example, this subdivision can add vertices equal to the number of vertices thinned out from the original mesh. That is, subdivision can be used to maintain the same number of vertices as the original mesh. In this specification, these added vertices are also referred to as division points. This subdivision can also be repeated recursively. For example, in a technique called midpoint, the process of adding vertices to the midpoints of edges (subdivision) is repeated recursively. In other words, recursive subdivision increases the number of vertices and improves the resolution of the mesh. In this way, it is possible to perform subdivision up to any desired level of resolution (i.e., control the resolution of the subdivided mesh). In other words, the subdivided mesh can be layered according to its level of resolution. In other words, this can be considered a layering of the subdivision process and the vertices (division points) and edges obtained by the subdivision process.

[0025] However, the connections of the base mesh are updated when the vertices of the original mesh are thinned out. Therefore, the division points obtained by subdivision are formed on these updated connections (edges). As a result, the shape of the subdivided base mesh differs from the shape of the original mesh. More specifically, as shown in the bottom part of Figure 2, the positions of the division points (on the dotted line) differ from those of the original mesh. In addition, the positions of the vertices of the base mesh may differ from those of the original mesh.

[0026] In other words, by moving the positions of the vertices of the subdivided base mesh (the vertices or division points of the base mesh) closer to the vertex positions of the original mesh, the difference in shape between the subdivided base mesh and the original mesh can be reduced. In this specification, such movement of the vertices of the subdivided base mesh (the vertices or division points of the base mesh) is also referred to as displacement. Furthermore, the amount and direction of this displacement, expressed as a vector, is also referred to as a displacement vector. Ideally, by displacing each vertex of the subdivided base mesh, the shape of the subdivided base mesh can be made to match the shape of the original mesh. In other words, the original mesh can be expressed as a base mesh and a displacement vector.

[0027] In V-DMC, such base meshes and displacement vectors are coded instead of the original mesh (geometry). By coding the base meshes and displacement vectors in this way, it is possible to code with a reduced number of polygons (i.e., the number of vertices and edges) compared to coding the original mesh, which generally reduces the amount of code for the same quality. In other words, it is possible to improve coding efficiency.

[0028] During decoding, as described above, a mesh is restored (generated) by subdividing a base mesh and applying a displacement vector to each vertex of the subdivided base mesh to displace it. In this specification, this restored mesh is also referred to as a restored mesh. Ideally, a restored mesh equivalent to the original mesh can be generated. Note that although the shape of the polygon (face) may be any polygonal shape, the following description will be given assuming that the polygon is triangular. Therefore, in the following, a polygon (face) will also be referred to as a triangle.

[0029] <Encoding and Decoding of V-DMC Data> In the case of V-DMC, mesh data consists of a base mesh, displacement vectors, attributes, and atlas information. This data group is also referred to as V-DMC data. The base mesh consists of information indicating vertices and connections, and is encoded using an existing mesh encoding method such as Draco.

[0030] Displacement vectors are converted into displacement coefficients using a predetermined method. These displacement coefficients are arranged as pixel values ​​in a two-dimensional area (also called a displacement map). This arrangement (mapping) of displacement coefficients is also called packing. A moving image (also called a displacement video) with the displacement map as its frame image is coded using a coding method for 2D moving images. In other words, the displacement coefficients are scalar values ​​corresponding to the displacement vectors. A displacement map is map information (also called image data) that stores the displacement coefficients as pixel values. A displacement video is moving image data with the displacement map as its frame image.

[0031] An attribute is non-geometry information applied to a mesh (geometry), which is 3D data. For example, an attribute may include a texture applied to a face of the mesh (geometry). The attribute (e.g., texture) is divided into multiple subregions, each of which is projected in a predetermined projection direction, and the projected images (patches) are arranged in a two-dimensional region (also called an attribute map). In other words, attribute patches are packed into the attribute map. A video (also called attribute video) using the attribute map as frame images is encoded using a 2D video encoding method. In other words, the attribute map is map information (also called image data) that stores the patches (projected textures) as pixel values. Attribute video is video data using the attribute map as frame images.

[0032] Atlas information is information used when reconstructing a mesh. For example, atlas information may include correspondence between the base mesh and a displacement map or attribute map (such as a UV map), quantized values ​​of displacement vectors, etc. This atlas information is encoded using a predetermined encoding method.

[0033] The coded data (bitstream) of each data is decoded by a decoding method corresponding to the coding method. In other words, by decoding the coded data (bitstream), various information such as base meshes, displacement vectors, attributes, and atlas information is restored (generated).

[0034] <Initial Edge List> As described above, in the V-DMC decoding process, when a base mesh is subdivided, edges are divided and division points are added. For this subdivision, an initial edge list indicating each edge of the base mesh is derived. The initial edge list is generated from the triangle list of the base mesh. The triangle list is a list of triangles that make up the base mesh, and each triangle is represented by its vertex identifier. In other words, based on this triangle list, the edge of each triangle (i.e., each edge of the base mesh) can be represented as a pair (combination) of the identifiers of each vertex.

[0035] 3, in the decoding process, the encoded data of the base mesh (Base mesh stream) is decoded (decode Draco), and then an initial edge list is generated using the triangle list of the base mesh (initialize edge list), and the base mesh is then subdivided (Mesh Subdivision) using the initial edge list.

[0036] The geometry video stream is decoded and unpacked to extract displacement vectors, which are then applied to the refined base mesh to reconstruct a decoded mesh.

[0037] In generating such an initial edge list, as shown in FIG. 3, combinations of adjacent vertices are generated (compute neighbors), and an initial edge list is generated using these combinations (generate unique edge list).

[0038] Here, the initial edge list is a unique edge list that represents the edges of the base mesh by non-overlapping combinations. In other words, in the initial edge list, each edge is represented by a combination (pair) of adjacent vertices, but these combinations do not overlap. In contrast, when generating combinations (pairs) of adjacent vertices from a triangle list, multiple identical combinations may be generated. For example, if two triangles meet at an edge, that edge is shared by those triangles. Therefore, the combination representing that edge (i.e., the "combination of adjacent vertices") is generated as the edge of each triangle. In other words, multiple combinations representing the same edge are generated (i.e., overlapping combinations).

[0039] Therefore, it is necessary to remove overlapping combinations (combinations that have already been registered). For example, suppose an initial edge list for a base mesh is to be generated as shown in FIG. 4. In this case, as shown in FIG. 5, a condition such as "left < right" is added in the ComputeNeighbors process to generate an edge list without overlapping. This edge list without overlapping is then used as the initial edge list.

[0040] In other words, with this method, it was necessary to generate combinations (pairs) of adjacent vertices from the triangle list, and for each combination, determine whether it overlaps with other combinations (edges already registered in the initial edge list).

[0041] 6 is a diagram showing an example of a pseudo program for the process of deriving an initial edge list. As shown in this example, the ComputeNeighbors process is executed. An example of the pseudo program for the ComputeNeighbors process is shown in FIG. 7. As shown in this example, in the ComputeNeighbors process, a duplication determination (FindNeighbor) is performed for each combination.

[0042] For example, as shown in Figure 8, the first combination (i=0) (0,2) of the first triangle (t=0) in the triangle list is processed. Since this is the first combination, there are no duplicate combinations, so the vertex with identifier "2" is registered in the list neighbors[0] indicating vertices connected to the vertex with identifier "0", and the vertex with identifier "0" is registered in the list neighbors[2] indicating vertices connected to the vertex with identifier "2".

[0043] Next, as shown in Figure 9, the second combination (i=1) of the first triangle (t=0) in the triangle list (2,1) is processed. As a result of the overlap determination, no overlapping combinations exist, so the vertex with identifier "2" is registered in the list neighbors[1] indicating vertices connected to the vertex with identifier "1", and the vertex with identifier "1" is registered in the list neighbors[2] indicating vertices connected to the vertex with identifier "2".

[0044] Next, as shown in Figure 10, the third combination (i=2) of the first triangle (t=0) in the triangle list is processed. As a result of the overlap determination, no overlapping combinations exist, so the vertex with identifier "1" is registered in the list neighbors[0] indicating vertices connected to the vertex with identifier "0", and the vertex with identifier "0" is registered in the list neighbors[1] indicating vertices connected to the vertex with identifier "1".

[0045] Next, as shown in Figure 11, the first combination (i=0) of the second triangle (t=1) in the triangle list (3,2) is processed. As a result of the overlap determination, no overlapping combinations exist, so the vertex with identifier "3" is registered in the list neighbors[2] indicating vertices connected to the vertex with identifier "2", and the vertex with identifier "2" is registered in the list neighbors[3] indicating vertices connected to the vertex with identifier "3".

[0046] Next, the second triangle (t=1) in the triangle list (i=1) is processed as a combination (2,0) as shown in Figure 12. As a result of the overlap determination, it overlaps with the combination (0,2) being processed in Figure 8, so registration of this combination in the list neighbors[] is skipped.

[0047] As shown in Figure 13, the above process is repeated for each combination up to the third pair (i=2) of the third triangle (t=2) in the triangle list (0,4). A non-overlapping edge list is generated from the list neighbors[] thus generated.

[0048] As described above, overlap determination is performed for each combination. In this overlap determination, a search is performed for combinations that overlap with the combination to be processed, which results in a large load. As a result, there is a risk of the load of the decoding process increasing. Generally, as the number of edges increases, the search range for each overlap determination becomes wider, and the load increases. Furthermore, as the number of edges increases, the number of overlap determination processes also increases. Therefore, the load of the overlap determination process tends to increase as the number of edges increases. Therefore, there is a risk of the load of the decoding process increasing as the size (number of edges) of the base mesh increases.

[0049] For example, in one implementation example, within the decoding process, the processes shown in gray in Figure 3 (e.g., decode Draco, initialize edge list (compute neighbors, generate unique edge list)) can be assumed to be executed by a CPU (Central Processing Unit), the processes shown in bold frames (e.g., decode displacement, mesh subdivision, mesh position refinement) can be executed by a CPU or a GPU (Graphics Processing Unit), and the processes shown in thin frames (e.g., decode geometry video) can be assumed to be executed by hardware. GPUs are provided as resources separate from the CPU, but are specialized for pipeline processing and parallel processing.

[0050] In contrast, CPUs are more versatile than GPUs, but they can also be provided with resources for purposes other than decoding. Therefore, depending on the situation, it is conceivable that the CPU resources allocated to decoding may be significantly reduced. In such cases, it may not be possible to secure sufficient CPU resources to handle the load of processes executed by the CPU, such as the initial edge list generation process, and these processes may become a bottleneck, resulting in delays in the decoding process.

[0051] 3. Transmission of Initial Edge List Therefore, the initial edge list may be generated on the encoding side and supplied to the decoding side. That is, as shown in Fig. 14, the initial edge list may be provided to the decoding process as atlas information.

[0052] For example, the first information processing device may include an initial edge list generation unit that generates an initial edge list, which is a unique edge list that indicates the edges of a base mesh by non-overlapping combinations of adjacent vertices of the base mesh; an atlas information encoding unit that encodes atlas information including the initial edge list and generates encoded data of the atlas information; and a bitstream generation unit that generates a bitstream including the encoded data of the base mesh, encoded data of the displacement video, encoded data of the attribute video, and encoded data of the atlas information.

[0053] In addition, in the first information processing device, an initial edge list, which is a unique edge list that indicates the edges of a base mesh by non-overlapping combinations of adjacent vertices of the base mesh, is generated, atlas information including the initial edge list is encoded, encoded data of the atlas information is generated, and a bitstream including encoded data of the base mesh, encoded data of the displacement video, encoded data of the attribute video, and encoded data of the atlas information is generated.

[0054] For example, the second information processing device may include a demultiplexing unit that demultiplexes the bitstream and extracts coded data of the base mesh, coded data of the displacement video, coded data of the attribute video, and coded data of the atlas information, an atlas information decoding unit that decodes the coded data of the extracted atlas information, a base mesh decoding unit that decodes the coded data of the extracted base mesh, and a subdivision unit that subdivides the base mesh using an initial edge list included in the atlas information, which is a unique edge list that indicates the edges of the base mesh by non-overlapping combinations of adjacent vertices of the base mesh.

[0055] For example, in a second information processing device, the bitstream may be demultiplexed to extract the coded data of the base mesh, the coded data of the displacement video, the coded data of the attribute video, and the coded data of the atlas information, the coded data of the extracted atlas information may be decoded, the coded data of the extracted base mesh may be decoded, and the base mesh may be subdivided using an initial edge list included in the atlas information, which is a unique edge list that indicates the edges of the base mesh by non-overlapping combinations of adjacent vertices of the base mesh.

[0056] In the first information processing device and the second information processing device, the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of the object, and is generated by thinning out vertices from the original mesh. The displacement video is a moving image whose frame images are a displacement map, which is a two-dimensional area in which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed. The attribute video is a moving image whose frame images are an attribute map, which is a two-dimensional area in which the texture of the original mesh is packed. The atlas information is information used for mesh reconstruction, including information indicating the correspondence between the base mesh and the displacement map and attribute map.

[0057] This eliminates the need for the process of generating an initial edge list (initialize edge list (Figure 3)), and allows for the base mesh to be subdivided (Mesh Subdivision) after decoding the base mesh (decode Draco). In other words, it is possible to skip processes such as generating combinations of adjacent vertices (compute neighbors) and generating an initial edge list using those combinations (generate unique edge list), as shown in Figure 3.

[0058] Therefore, the above-described overlap determination process is unnecessary, and the increase in the load of the decoding process can be suppressed accordingly. This suppresses the increase in the load of the process executed by the CPU, and therefore suppresses the occurrence of delays in the decoding process due to a lack of CPU resources.

[0059] <Encoding device> The present technology can be applied to any device. For example, the present technology can be applied to an encoding device that encodes a mesh. Fig. 15 is a block diagram showing an example of the configuration of an encoding device that is one aspect of an information processing device to which the present technology is applied. The encoding device 100 (first information processing device) shown in Fig. 15 is a device that encodes a mesh.

[0060] Fig. 15 shows the main processing units, data flows, etc., but does not necessarily show everything. In other words, in encoding device 100, there may be processing units that are not shown as blocks in Fig. 15, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 15.

[0061] The encoding device 100 encodes a mesh using a method essentially similar to the V-DMC method described in the aforementioned non-patent document, except that the present technology is applied. For example, the encoding device 100 acquires an original mesh to be encoded and an attribute map containing a texture corresponding to the original mesh. Note that the original mesh contains not only mesh geometry information but also information indicating the correspondence with the attribute map (e.g., a UV list). The encoding device 100 encodes the original mesh and attribute map using the V-DMC method, generates a V-DMC bitstream, and outputs it.

[0062] 15 , the encoding device 100 (first information processing device) has a pre-processing unit 111 and an encoding unit 112. The pre-processing unit 111 performs pre-processing before encoding. As shown in FIG. 15 , the pre-processing unit 111 has a mesh decimation unit 121, an atlas information generation unit 122, an initial edge list generation unit 123, and a displacement vector generation unit 124.

[0063] The mesh decimation unit 121 performs processing related to mesh decimation. For example, the mesh decimation unit 121 may obtain an original mesh to be input to the encoding device 100. The mesh decimation unit 121 may also perform decimation processing (thinning out vertices) on the original mesh to generate a base mesh. The mesh decimation unit 121 may supply the generated base mesh together with the original mesh to the atlas information generation unit 122.

[0064] The atlas information generation unit 122 performs processing related to the generation of atlas information corresponding to a base mesh. For example, the atlas information generation unit 122 may acquire a base mesh or an original mesh supplied from the mesh decimation unit 121. The atlas information generation unit 122 may generate atlas information by, for example, UV unwrapping the base mesh. The atlas information generation unit 122 may exchange information with the initial edge list generation unit 123. The atlas information generation unit 122 may supply the generated atlas information to the displacement vector generation unit 124 together with the base mesh, etc.

[0065] The initial edge list generation unit 123 executes processing related to the generation of an initial edge list. For example, the initial edge list generation unit 123 may acquire a base mesh to be processed by the atlas information generation unit 122. The initial edge list generation unit 123 may use the base mesh to generate an initial edge list. For example, the initial edge list generation unit 123 may use the triangle list of the base mesh to execute processing such as generating combinations of adjacent vertices (compute neighbors) and generating an initial edge list using the combinations (generate unique edge list). The initial edge list generation unit 123 may supply the initial edge list generated as described above to the atlas information generation unit 122 as atlas information.

[0066] The displacement vector generation unit 124 performs processing related to the generation of displacement vectors. For example, the displacement vector generation unit 124 may acquire a base mesh, atlas information, etc. supplied from the atlas information generation unit 122. Alternatively, the displacement vector generation unit 124 may acquire an original mesh input to the encoding device 100. Alternatively, the displacement vector generation unit 124 may use the acquired information to generate displacement vectors that displace vertices of the subdivided base mesh. The displacement vector generation unit 124 may supply the generated displacement vectors to the encoding unit 112 together with the base mesh, atlas information (including the initial edge list), etc.

[0067] The encoding unit 112 performs processing related to encoding of V-DMC data. For example, the encoding unit 112 may acquire an original mesh input to the encoding device 100. The encoding unit 112 may also acquire a base mesh, a displacement vector, atlas information, and the like, supplied from the displacement vector generation unit 124. The encoding unit 112 may also acquire an attribute map input to the encoding device 100. The encoding unit 112 may use this information to encode the atlas information, the base mesh, the displacement vector, and the attribute map, respectively, and generate the respective encoded data. The encoding unit 112 may also multiplex these encoded data as substreams to generate a single bitstream. This bitstream is also referred to as a V-DMC bitstream. The encoding unit 112 may output the generated V-DMC bitstream to the outside of the encoding device 100.

[0068] <Encoding Unit> Fig. 16 is a block diagram showing an example of the main configuration of the encoding unit 112. Note that Fig. 16 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, the encoding unit 112 may include processing units that are not shown as blocks in Fig. 16, or processes or data flows that are not shown as arrows, etc. in Fig. 16.

[0069] As shown in FIG. 16, the encoding unit 112 has an atlas information encoding unit 151, a base mesh encoding unit 152, a displacement vector correction unit 153, a displacement video encoding unit 154, a mesh reconstruction unit 155, an attribute map conversion unit 156, an attribute video encoding unit 157, and a multiplexing unit 158.

[0070] The atlas information encoding unit 151 performs processing related to encoding of atlas information. For example, the atlas information encoding unit 151 may acquire atlas information supplied from the displacement vector generation unit 124. This atlas information may include, for example, information such as an initial edge list. Furthermore, the atlas information encoding unit 151 may encode the acquired atlas information using a predetermined encoding method to generate encoded data of the atlas information. The atlas information encoding unit 151 may supply the generated encoded data of the atlas information to the multiplexing unit 158.

[0071] The base mesh encoding unit 152 performs processing related to encoding of the base mesh. For example, the base mesh encoding unit 152 may acquire the base mesh supplied from the displacement vector generation unit 124. The base mesh encoding unit 152 may quantize the acquired base mesh and encode it using a predetermined encoding method (e.g., Draco) to generate encoded data of the base mesh. The base mesh encoding unit 152 may encode the base mesh in accordance with control information included in the atlas information. The base mesh encoding unit 152 may supply the generated encoded data of the base mesh to the displacement vector correction unit 153. The base mesh encoding unit 152 may supply the generated encoded data of the base mesh to the multiplexing unit 158.

[0072] The displacement vector correction unit 153 performs processing related to the correction of the displacement vector. For example, the displacement vector correction unit 153 may acquire a base mesh and a displacement vector supplied from the displacement vector generation unit 124. Alternatively, the displacement vector correction unit 153 may acquire encoded data of the base mesh supplied from the base mesh encoding unit 152. The displacement vector correction unit 153 may correct the displacement vector based on this information. For example, the displacement vector correction unit 153 may decode the acquired encoded data of the base mesh, compare the base mesh before and after encoding to determine encoding distortion of the base mesh, and correct the displacement vector in accordance with the encoding distortion. The displacement vector correction unit 153 may supply the corrected displacement vector to the displacement video encoding unit 154. Alternatively, the displacement vector correction unit 153 may dequantize the decoded base mesh and supply it to the mesh reconstruction unit 155.

[0073] The displacement video encoding unit 154 performs processing related to encoding of the displacement video. The displacement video is a moving image having a displacement map, which is a two-dimensional area in which displacement vectors are packed, as frame images. For example, the displacement video encoding unit 154 may acquire displacement vectors supplied from the displacement vector correction unit 153. The displacement video encoding unit 154 may also generate a displacement map by wavelet transforming the displacement vectors, quantizing them, and packing them into a two-dimensional area. The displacement video encoding unit 154 may also generate a displacement video having the displacement map as frame images. The displacement video encoding unit 154 may also encode the generated displacement video using a predetermined encoding method for 2D moving images to generate encoded data of the displacement video. The displacement video encoding unit 154 may also supply the generated encoded data of the displacement video to the multiplexing unit 158. The displacement video encoding unit 154 may also decode the generated encoded data of the displacement video, unpack the displacement vectors from the displacement map, and dequantize the displacement vectors. The displacement video encoding unit 154 may provide the dequantized displacement vectors to the mesh reconstruction unit 155 .

[0074] The mesh reconstruction unit 155 performs processing related to mesh reconstruction. For example, the mesh reconstruction unit 155 may acquire a base mesh supplied from the displacement vector correction unit 153. The mesh reconstruction unit 155 may also acquire a displacement vector supplied from the displacement video encoding unit 154. The mesh reconstruction unit 155 may use these to reconstruct a mesh. The mesh reconstruction unit 155 may supply the reconstructed mesh to the attribute map conversion unit 156.

[0075] The attribute map conversion unit 156 performs processing related to attribute map conversion. For example, the attribute map conversion unit 156 may acquire a reconstructed mesh supplied from the mesh reconstruction unit 155. Alternatively, the attribute map conversion unit 156 may acquire atlas information supplied from the displacement vector generation unit 124. Alternatively, the attribute map conversion unit 156 may acquire an original mesh and an attribute map input to the encoding device 100. The attribute map conversion unit 156 may convert the acquired attribute map based on other acquired information. For example, the attribute map conversion unit 156 may convert the attribute map based on the atlas information, the original mesh, or the like so that it corresponds to the reconstructed mesh. In other words, the attribute map conversion unit 156 can be said to generate a converted attribute map. Therefore, the attribute map conversion unit 156 can also be said to be an attribute map generation unit. The attribute map conversion unit 156 may supply the converted attribute map to the attribute video encoding unit 157 .

[0076] The attribute video encoding unit 157 performs processing related to encoding of the attribute video. For example, the attribute video encoding unit 157 may acquire an attribute map supplied from the attribute map conversion unit 156. The attribute video encoding unit 157 may also generate attribute video using the acquired attribute map as frame images. The attribute video encoding unit 157 may also encode the generated attribute video using a predetermined encoding method for 2D moving images to generate encoded data of the attribute video. The attribute video encoding unit 157 may also supply the generated encoded data of the attribute video to the multiplexing unit 158.

[0077] The multiplexing unit 158 ​​performs processing related to multiplexing of encoded data (substreams). For example, the multiplexing unit 158 ​​may acquire encoded data of atlas information supplied from the atlas information encoding unit 151. Alternatively, the multiplexing unit 158 ​​may acquire encoded data of base meshes supplied from the base mesh encoding unit 152. Alternatively, the multiplexing unit 158 ​​may acquire encoded data of displacement video supplied from the displacement video encoding unit 154. Alternatively, the multiplexing unit 158 ​​may acquire encoded data of attribute video supplied from the attribute video encoding unit 157. The multiplexing unit 158 ​​may multiplex these pieces of encoded data as substreams to generate a V-DMC bitstream. Therefore, the multiplexing unit 158 ​​can also be referred to as a bitstream generation unit. The multiplexing unit 158 ​​may output the generated V-DMC bitstream to an external device from the encoding device 100. For example, the multiplexing unit 158 ​​may supply the V-DMC bitstream to a decoding device 200, which will be described later. Therefore, the multiplexer 158 can also be said to be a supplier (provider) of the V-DMC bitstream.

[0078] That is, in the encoding device 100 (first information processing device), the initial edge list generation unit 123 generates an initial edge list, which is a unique edge list that indicates the edges of a base mesh by non-overlapping combinations of adjacent vertices of the base mesh. The atlas information encoding unit 151 encodes atlas information including the initial edge list to generate encoded data of the atlas information. The multiplexing unit 158 ​​generates a bitstream including the encoded data of the base mesh, the encoded data of the displacement video, the encoded data of the attribute video, and the encoded data of the atlas information.

[0079] With this configuration, the encoding device 100 can suppress an increase in the load of the decoding process, as described above, and thereby suppress delays in the decoding process due to a lack of CPU resources.

[0080] <Flow of Encoding Process> An example of the flow of the encoding process executed by the encoding device 100 will be described with reference to the flowchart of FIG.

[0081] When the encoding process starts, in step S101, the mesh decimation unit 121 of the encoding device 100 decimates the original mesh to be encoded to generate a base mesh.

[0082] In step S102, the atlas information generating unit 122 generates atlas information for the base mesh.

[0083] In step S103, the initial edge list generating unit 123 generates an initial edge list, which is a unique edge list that indicates the edges of the base mesh by non-overlapping combinations of adjacent vertices of the base mesh.

[0084] In step S104, the displacement vector generating unit 124 generates a displacement vector.

[0085] In step S105, the encoding unit 312 performs V-DMC encoding processing on the V-DMC data generated as described above to generate a V-DMC bitstream.

[0086] The encoding process ends when the process of step S305 ends. The encoding device 100 executes such encoding process for each frame of the original mesh.

[0087] <Flow of V-DMC Encoding Process> Next, an example of the flow of the V-DMC encoding process executed in step S105 of FIG. 17 will be described with reference to the flowchart of FIG.

[0088] When the V-DMC encoding process starts, in step S121, the atlas information encoding unit 151 encodes atlas information including an initial edge list.

[0089] In step S122, the base mesh encoding unit 152 encodes the base mesh.

[0090] In step S123, the displacement vector correction unit 153 corrects the displacement vector.

[0091] In step S124, the disparity video encoding unit 154 encodes a disparity video in which the disparity map in which the corrected disparity vectors are packed is used as a frame image.

[0092] In step S125, the mesh reconstructing unit 155 reconstructs the mesh.

[0093] In step S126, the attribute map conversion unit 156 converts the attribute map.

[0094] In step S127, the attribute video encoding unit 157 encodes the attribute video using the attribute map as a frame image.

[0095] In step S128, the multiplexing unit 158 ​​multiplexes the coded data of the atlas information, the coded data of the bitstream, the coded data of the displacement video, and the coded data of the attribute video to generate a V-DMC bitstream.

[0096] When the process of step S128 ends, the V-DMC encoding process ends, and the process returns to FIG.

[0097] By performing the above-described processes, the encoding device 100 can suppress an increase in the load of the decoding process, as described above, thereby suppressing delays in the decoding process due to a lack of CPU resources.

[0098] <Decoding device> The present technology can be applied to a decoding device that decodes mesh encoded data. Fig. 19 is a block diagram showing an example of the configuration of a decoding device that is one aspect of an information processing device to which the present technology is applied. The decoding device 200 (second information processing device) shown in Fig. 19 is a device that decodes mesh encoded data (V-DMC bitstream generated by the multiplexing unit 158 ​​(Fig. 16)) generated in the encoding device 100 (Fig. 15), for example.

[0099] Fig. 19 shows the main processing units, data flows, etc., but does not necessarily include all of them. In other words, in the decoding device 200, there may be processing units that are not shown as blocks in Fig. 19, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 19.

[0100] The decoding device 200 decodes coded data of meshes that have been coded using a method essentially similar to the V-DMC method described in the aforementioned non-patent document, except that the present technology is applied. For example, the decoding device 200 obtains a V-DMC bitstream. This V-DMC bitstream may be generated by, for example, the coding device 100. The decoding device 200 decodes the V-DMC bitstream, reconstructs a mesh (also referred to as a decoded mesh), applies a texture to the decoded mesh, generates a display image for displaying the decoded mesh, and outputs the display image to an external device.

[0101] As shown in Figure 19, the decoding device 200 (second information processing device) has a demultiplexing unit 211, an atlas information decoding unit 212, a base mesh decoding unit 213, a subdivision unit 215, a displacement video decoding unit 216, an unpacking unit 217, a displacement vector application unit 218, an attribute video decoding unit 219, and a display processing unit 220.

[0102] The demultiplexing unit 211 performs demultiplexing processing. For example, the demultiplexing unit 211 may acquire a V-DMC bitstream to be decoded and supplied to the decoding device 200. The demultiplexing unit 211 may also demultiplex the acquired V-DMC bitstream to extract coded data of atlas information, coded data of base meshes, coded data of displacement video, and coded data of attribute video. Therefore, the demultiplexing unit 211 can also be considered an acquisition unit for the V-DMC bitstream or various information contained in the V-DMC bitstream. The demultiplexing unit 211 may supply the extracted coded data of atlas information to the atlas information decoding unit 212. The demultiplexing unit 211 may also supply the extracted coded data of base meshes to the base mesh decoding unit 213. The demultiplexing unit 211 may also supply the extracted coded data of displacement video to the displacement video decoding unit 216. Furthermore, the demultiplexing unit 211 may supply the extracted coded data of the attribute video to the attribute video decoding unit 219 .

[0103] The atlas information decoding unit 212 performs processing related to decoding of encoded data of atlas information. For example, the atlas information decoding unit 212 may acquire encoded data of atlas information supplied from the demultiplexing unit 211. The atlas information decoding unit 212 may also decode the acquired encoded data of atlas information to generate (restore) atlas information. The atlas information decoding unit 212 may supply the generated atlas information to the subdivision unit 215 and the display processing unit 220.

[0104] The base mesh decoding unit 213 performs processing related to decoding of the coded data of the base mesh. For example, the base mesh decoding unit 213 may acquire the coded data of the base mesh supplied from the demultiplexing unit 211. The base mesh decoding unit 213 may also decode the acquired coded data of the base mesh using a predetermined decoding method (e.g., Draco, etc.) to generate (restore) the base mesh (e.g., a vertex list, a triangle list, etc.). The base mesh decoding unit 213 may supply the generated base mesh to the subdivision unit 215.

[0105] The subdivision unit 215 performs processing related to subdivision of triangles of a base mesh. For example, the subdivision unit 215 may acquire a base mesh supplied from the base mesh decoding unit 213. The subdivision unit 215 may acquire atlas information supplied from the atlas information decoding unit 212. This atlas information may include an initial edge list. The subdivision unit 215 may subdivide the base mesh (triangles thereof) and generate division points. In this case, the subdivision unit 215 may subdivide the base mesh using the initial edge list included in the atlas information. The subdivision unit 215 may supply the subdivided base mesh to the displacement vector application unit 218.

[0106] The displacement video decoding unit 216 performs processing related to decoding of encoded data of the displacement video. For example, the displacement video decoding unit 216 may acquire encoded data of the displacement video supplied from the demultiplexing unit 211. Furthermore, the displacement video decoding unit 216 may decode the acquired encoded data of the displacement video using a predetermined decoding method for 2D moving images to generate (restore) the displacement video. The displacement video decoding unit 216 may supply the generated displacement video to the unpacking unit 217.

[0107] The unpacking unit 217 performs processing related to unpacking of the displacement vector. For example, the unpacking unit 217 may acquire the displacement video supplied from the displacement video decoding unit 216. The unpacking unit 217 may also unpack the displacement vector from a displacement map, which is a frame image of the displacement video. The unpacking unit 217 may supply the displacement vector obtained in this way to the displacement vector application unit 218.

[0108] The displacement vector application unit 218 performs processing related to application of displacement vectors to the subdivided base mesh. For example, the displacement vector application unit 218 may obtain the subdivided base mesh supplied from the subdivision unit 215. Alternatively, the displacement vector application unit 218 may obtain the displacement vectors supplied from the unpacking unit 217. Alternatively, the displacement vector application unit 218 may apply the displacement vectors to the vertices of the subdivided base mesh. In other words, the displacement vector application unit 218 may generate a decoded mesh. The displacement vector application unit 218 may supply the generated decoded mesh to the display processing unit 220.

[0109] The attribute video decoding unit 219 executes processing related to decoding of the attribute video. For example, the attribute video decoding unit 219 may acquire coded data of the attribute video supplied from the demultiplexing unit 211. The attribute video decoding unit 219 may also decode the acquired coded data of the attribute video using a predetermined decoding method for 2D moving images to generate (restore) the attribute video. The attribute video decoding unit 219 may supply an attribute map, which is a frame image of the generated attribute video, to the display processing unit 220.

[0110] The display processing unit 220 performs processing related to mesh display. For example, the display processing unit 220 may acquire atlas information supplied from the atlas information decoding unit 212. The display processing unit 220 may also acquire a decoded mesh supplied from the displacement vector application unit 218. The display processing unit 220 may also acquire an attribute map supplied from the attribute video decoding unit 219. The display processing unit 220 may use the acquired atlas information to extract a texture from the attribute map and apply the texture to a face of the decoded mesh corresponding to the texture. In other words, the display processing unit 220 may attach the texture to the face. The display processing unit 220 may render the decoded mesh to which the texture has been applied and generate a display image for displaying the decoded mesh to which the texture has been applied. The display processing unit 220 may then supply the generated display image to an external device outside the decoding device 200, causing the display image to be displayed by another device or the like.

[0111] That is, in the decoding device 200 (second information processing device), the demultiplexing unit 211 demultiplexes the bitstream and extracts coded data of the base mesh, coded data of the displacement video, coded data of the attribute video, and coded data of the atlas information. The atlas information decoding unit 212 decodes the coded data of the extracted atlas information. The base mesh decoding unit 213 decodes the coded data of the extracted base mesh. The subdivision unit 215 subdivides the base mesh using an initial edge list, which is a unique edge list included in the atlas information and indicates the edges of the base mesh by non-overlapping combinations of adjacent vertices of the base mesh.

[0112] With this configuration, the decoding device 200 can suppress an increase in the load of the decoding process, as described above, and thereby suppress delays in the decoding process due to a lack of CPU resources.

[0113] <Flow of Decoding Process> An example of the flow of the decoding process executed by the decoding device 200 will be described with reference to the flowchart of FIG.

[0114] When the decoding process starts, the demultiplexing unit 211 of the decoding device 200 demultiplexes the V-DMC bitstream in step S201.

[0115] In step S202, the atlas information decoding unit 212 decodes the coded data of the atlas information to generate (restore) the atlas information. This atlas information includes the initial edge list.

[0116] In step S203, the base mesh decoding unit 213 decodes the coded data of the base mesh to generate (restore) the base mesh.

[0117] In step S204, the subdivision unit 215 subdivides the base mesh using the initial edge list.

[0118] In step S205, the displaced video decoding unit 216 decodes the coded data of the displaced video to generate (restore) the displaced video.

[0119] In step S206, the unpacking unit 217 unpacks the displacement map, which is the frame images of the displacement video, and extracts the displacement vectors.

[0120] In step S207, the displacement vector application unit 218 applies the unpacked displacement vectors to the vertices of the subdivided base mesh to generate a decoded mesh.

[0121] In step S208, the attribute video decoding unit 219 decodes the coded data of the attribute video to generate (restore) the attribute video.

[0122] In step S210, the display processing unit 220 uses the atlas information to apply a texture extracted from an attribute map, which is a frame image of the attribute video, to the faces of the generated decoded mesh, and renders the decoded mesh to generate a display image.

[0123] When the process of step S210 is completed, the decoding process ends. The decoding device 200 executes such a decoding process for each frame of the original mesh.

[0124] By performing the processes as described above, the decoding device 200 can suppress an increase in the load of the decoding process, as described above, and thereby suppress delays in the decoding process due to a lack of CPU resources.

[0125] <Encoded Data of Atlas Information> With this method, an initial edge list is transmitted as described above. The initial edge list is, for example, a non-duplicate edge list as shown in FIG. 5. Therefore, there is a risk that the amount of code in the V-DMC bitstream will increase by the amount of the initial edge list. For example, the number of non-duplicate edges is 1.5 num_triangle to 3 num_triangle. Here, num_triangle indicates the number of triangles. Note that "A_B" indicates that B is a subscript of A. In other words, num_triangle indicates that "triangle" is a subscript of "num." The number of bits required per edge is 2 log(num_vertex). Here, num_vertex indicates the number of vertices. num_vertex indicates that "vertex" is a subscript of "num." Therefore, the total data size of the initial edge list is 3 num_triangle * log(num_vertex) to 6 num_triangle * log(num_vertex) [bits]. By transmitting the initial edge list, this amount of code is added to the V-DMC bitstream.

[0126] In other words, as the size (number of vertices) of the base mesh increases, the amount of code for the initial edge list increases, which could lead to an increase in the amount of code for the V-DMC bitstream. This could result in an increase in transmission bandwidth usage. Furthermore, an increase in the amount of code for the V-DMC bitstream increases the load on the decoding process, which could lead to delays in the decoding process due to a lack of CPU resources.

[0127] <4. Transmission of Edge Preservation Information> <Method 1> Therefore, as shown in the top row of the table in Figure 21, edge preservation information indicating whether each combination of adjacent vertices of the base mesh should be included as an edge in the initial edge list may be transmitted (Method 1).

[0128] For example, the third information processing device may include an edge preservation information generation unit that generates edge preservation information indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list, an atlas information encoding unit that encodes atlas information including the edge preservation information and generates encoded data of the atlas information, and a bitstream generation unit that generates a bitstream including the encoded data of the base mesh, the encoded data of the displacement video, the encoded data of the attribute video, and the encoded data of the atlas information.

[0129] For example, in a third information processing device, edge preservation information indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list may be generated, atlas information including the edge preservation information may be encoded, encoded data of the atlas information may be generated, and a bitstream including encoded data of the base mesh, encoded data of the displacement video, encoded data of the attribute video, and encoded data of the atlas information may be generated.

[0130] For example, a fourth information processing device may include a demultiplexing unit that demultiplexes a bitstream and extracts coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information; an atlas information decoding unit that decodes the coded data of the extracted atlas information; a base mesh decoding unit that decodes the coded data of the extracted base mesh; an initial edge list generation unit that generates an initial edge list of the base mesh using edge preservation information included in the atlas information that indicates whether each combination of adjacent vertices of the base mesh should be included as an edge in the initial edge list; and a subdivision unit that subdivides the base mesh using the initial edge list.

[0131] For example, in a fourth information processing device, the bit stream may be demultiplexed, and the encoded data of the base mesh, the encoded data of the displacement video, the encoded data of the attribute video, and the encoded data of the atlas information may be extracted, the extracted encoded data of the atlas information may be decoded, the extracted encoded data of the base mesh may be decoded, and an initial edge list of the base mesh may be generated using edge preservation information included in the atlas information that indicates whether each combination of adjacent vertices of the base mesh should be included as an edge in the initial edge list, and the base mesh may be subdivided using the initial edge list.

[0132] In the third and fourth information processing devices, the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh. The displacement video is a moving image whose frame images are a displacement map, which is a two-dimensional area packed with displacement vectors that indicate the displacement of vertices of a subdivided base mesh. The attribute video is a moving image whose frame images are an attribute map, which is a two-dimensional area packed with the texture of the original mesh. The atlas information is information used for mesh reconstruction, including information indicating the correspondence between the base mesh and the displacement map and attribute map. The initial edge list is a unique edge list that indicates the edges of the base mesh by non-overlapping combinations.

[0133] This simplifies the process of generating the initial edge list (initialize edge list (FIG. 3)) and makes the overlap determination process unnecessary. Therefore, the increase in the load of the decoding process can be suppressed accordingly. This suppresses the increase in the load of the process executed by the CPU, and therefore suppresses the occurrence of delays in the decoding process due to a lack of CPU resources.

[0134] Note that this edge preservation information may indicate in any manner whether each combination of adjacent vertices of the base mesh is to be included as an edge in the initial edge list. For example, the edge preservation information may be configured with one-bit flag information corresponding to each edge (each combination of adjacent vertices of the base mesh). For example, if the flag information is true (e.g., value "1"), it may indicate that the edge corresponding to that flag is to be registered in the initial edge list, and if the flag information is false (e.g., value "0"), it may indicate that the edge corresponding to that flag is not to be registered in the initial edge list. In other words, the edge preservation information can be considered control information that prevents overlapping edges (each combination of adjacent vertices of the base mesh) from being registered in the initial edge list.

[0135] Furthermore, as described above, the size of the initial edge list was 3num_triangle * log(num_vertex) to 6num_triangle * log(num_vertex) [bits] in total. However, when the edge preservation information is configured as described above, the data volume is only 3 bits per triangle, so the total size is approximately 1 / 2 log(num_vertex) to 1 / log(num_vertex) compared to the initial edge list. Therefore, the increase in the amount of code in the V-DMC bitstream can be suppressed compared to when the initial edge list is transmitted. This also reduces the increase in transmission bandwidth usage. Furthermore, the increase in the decoding load due to the increase in the amount of code in the V-DMC bitstream can be suppressed, and delays in the decoding process due to a lack of CPU resources can be suppressed.

[0136] Furthermore, this edge preservation information may be generated in any manner. For example, the edge preservation information may be generated using a triangle list of the base mesh. The triangle list is a list of identifiers for each vertex that constitutes each triangle of the base mesh. In other words, the triangle list indicates the identifiers for each vertex of the base mesh for each triangle. In other words, the edge preservation information may be generated as information indicating whether or not a combination of vertices that are considered to be adjacent to each other in this triangle list should be included as an edge in the initial edge list.

[0137] <Method 1-1> When Method 1 is applied, the edge preservation information may indicate the results of the overlap check for each edge in the ComputeNeighbors process (Method 1-1), as shown in the second row from the top of the table in Fig. 21. In other words, in the ComputeNeighbors process described with reference to Figs. 8 to 13, for example, the "search for combinations overlapping with the combination to be processed" may be skipped, and the edge preservation information may be used as the search result.

[0138] For example, in a third information processing device, the edge preservation information generation unit may perform ComputeNeighbors processing on a base mesh and generate edge preservation information using the results of a search for combinations that overlap with the ``combination of adjacent vertices of the base mesh'' being processed.

[0139] For example, in a fourth information processing device, the initial edge list generation unit may perform ComputeNeighbors processing using edge preservation information to generate Neighbors that indicate vertices adjacent to each vertex without overlapping, and then use the Neighbors to generate an initial edge list.

[0140] 22, edge preservation information is supplied to the decoding process along with the base mesh. Then, in the process of generating the initial edge list, the ComputeNeighbors process is executed using the edge preservation information. That is, the "search for combinations that overlap with the combination to be processed" in the ComputeNeighbors process is skipped, and the edge preservation information is applied as the search result.

[0141] For example, as shown in FIG. 23 , when the combination (0,2) of the first set (i=0) of the first triangle (t=0) in the triangle list is processed, the element of the edge preservation information corresponding to this combination (0,2) is “1” (true), so it is determined that no combinations overlapping this combination (0,2) are registered in neighbors, and this combination (0,2) is registered in neighbors. In other words, the vertex with identifier “2” is registered in the list neighbors[0] indicating vertices connected to the vertex with identifier “0,” and the vertex with identifier “0” is registered in the list neighbors[2] indicating vertices connected to the vertex with identifier “2.” As a result, the combination (0,2) is registered in the initial edge list.

[0142] In contrast, as shown in FIG. 24, when the combination (2,0) of the second set (i=1) of the second triangle (t=1) in the triangle list is processed, the element of the edge preservation information corresponding to this combination (2,0) is "0" (false), so it is determined that a combination that overlaps with this combination (2,0) is registered in neighbors. In fact, as explained in FIG. 23, the combination (0,2) is registered. Therefore, the registration of this combination (2,0) is skipped. As a result, the combination (2,0) is not registered in the initial edge list.

[0143] In this way, the edge preservation information can be used to control the registration of "combinations of adjacent vertices of the base mesh" in the initial edge list. Therefore, the "search for duplicate combinations" in the ComputeNeighbors process can be omitted (skip), and the increase in the load of the decoding process due to the search can be suppressed.

[0144] <Encoding device> Fig. 25 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which Method 1 (or Method 1-1) is applied. The encoding device 300 (third information processing device) shown in Fig. 25 is a device that encodes meshes, similar to the encoding device 100.

[0145] Fig. 25 shows the main processing units, data flows, etc., but is not limited to all that is shown in Fig. 25. In other words, in encoding device 300, there may be processing units that are not shown as blocks in Fig. 25, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 25.

[0146] As shown in FIG. 25, the encoding device 300 (third information processing device) has the same configuration as the encoding device 100 , and further includes an edge preservation information generation unit 323 .

[0147] The edge preservation information generation unit 323 executes processing related to the generation of edge preservation information. For example, the edge preservation information generation unit 323 may acquire a base mesh (triangle list) from the atlas information generation unit 122. The edge preservation information generation unit 323 may generate edge preservation information using the base mesh. For example, the edge preservation information generation unit 323 may execute a ComputeNeighbors process on the base mesh and generate edge preservation information using the results of a search for overlapping combinations. The edge preservation information generation unit 323 may supply the generated edge preservation information to the atlas information generation unit 122 as atlas information.

[0148] The other processing units basically perform the same processing as in the encoding device 100. The atlas information generation unit 122 may supply the atlas information including the edge preservation information to the displacement vector generation unit 124 together with the base mesh, etc. The configuration of the encoding unit 112 is the same as in the encoding device 100 (the example in FIG. 16 ).

[0149] With this configuration, the encoding device 300 can suppress an increase in the load of the decoding process. This allows the encoding device 300 to suppress an increase in the load of processing executed by the CPU, thereby suppressing delays in the decoding process due to a lack of CPU resources. Furthermore, the encoding device 300 can suppress an increase in the code amount of the V-DMC bitstream compared to when transmitting the initial edge list. Therefore, the encoding device 300 can suppress an increase in the usage of the transmission bandwidth. Furthermore, the encoding device 300 can suppress an increase in the load of the decoding process due to an increase in the code amount of the V-DMC bitstream, and suppress delays in the decoding process due to a lack of CPU resources. In other words, the encoding device 300 can suppress an increase in the load of the decoding process while suppressing an increase in the code amount.

[0150] <Flow of Encoding Process> An example of the flow of encoding process executed by the encoding device 300 will be described with reference to the flowchart of FIG.

[0151] When the encoding process is started, the processes of steps S301 and S302 are executed in the same manner as the processes of steps S101 and S102 (FIG. 17).

[0152] In step S303, the edge preservation information generation unit 323 generates edge preservation information based on the base mesh (triangle list). For example, the edge preservation information generation unit 323 may execute ComputeNeighbors processing on the base mesh and generate edge preservation information using the results of searching for overlapping combinations.

[0153] When the process of step S303 is completed, the process proceeds to step S304. The processes of steps S304 and S305 are executed in the same manner as the processes of steps S104 and S105 (FIG. 17). When the process of step S305 is completed, the encoding process ends. The encoding device 300 executes such encoding process for each frame of the original mesh.

[0154] The V-DMC encoding process executed in step S305 of FIG. 26 is executed in the same manner as in the example described with reference to the flowchart of FIG.

[0155] By performing each process as described above, the encoding device 300 can suppress an increase in the load of the decoding process. This allows the encoding device 300 to suppress an increase in the load of processes executed by the CPU, thereby suppressing delays in the decoding process due to a lack of CPU resources. Furthermore, the encoding device 300 can suppress an increase in the code amount of the V-DMC bitstream compared to when transmitting the initial edge list. Therefore, the encoding device 300 can suppress an increase in the transmission bandwidth usage. Furthermore, the encoding device 300 can suppress an increase in the load of the decoding process due to an increase in the code amount of the V-DMC bitstream, and suppress delays in the decoding process due to a lack of CPU resources. In other words, the encoding device 300 can suppress an increase in the load of the decoding process while suppressing an increase in the code amount.

[0156] <Decoding device> Fig. 27 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which Method 1 (or Method 1-1) is applied. Similar to the decoding device 200, the decoding device 400 (fourth information processing device) shown in Fig. 27 is a device that decodes, for example, coded data of a mesh generated in the coding device 300 (Fig. 25) (the V-DMC bitstream generated by the multiplexing unit 158 ​​(Fig. 16)).

[0157] Fig. 27 shows the main processing units, data flows, etc., but does not necessarily include all of them. In other words, in the decoding device 400, there may be processing units that are not shown as blocks in Fig. 27, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 27.

[0158] As shown in FIG. 27, a decoding device 400 (fourth information processing device) has the same configuration as the decoding device 200 , and further includes an initial edge list generation unit 414 .

[0159] The initial edge list generation unit 414 executes processing related to the generation of an initial edge list. For example, the initial edge list generation unit 414 may acquire a base mesh (including a triangle list, etc.) supplied from the base mesh decoding unit 213. The initial edge list generation unit 414 may also acquire atlas information supplied from the atlas information decoding unit 212. This atlas information includes edge preservation information. This edge preservation information may be generated by, for example, the encoding device 300.

[0160] The initial edge list generation unit 414 may generate an initial edge list using a base mesh (triangle list) and edge preservation information. For example, as described with reference to FIGS. 22 to 24 , the initial edge list generation unit 414 may execute a ComputeNeighbors process using the edge preservation information to generate Neighbors that indicate vertices adjacent to each vertex without overlapping, and then generate an initial edge list using the resulting Neighbors. In other words, the initial edge list generation unit 414 may skip the "search for combinations that overlap with the combination to be processed" step in the ComputeNeighbors process and apply the edge preservation information as the search result. The initial edge list generation unit 414 may supply the initial edge list generated in this manner to the subdivision unit 215 along with the base mesh, etc.

[0161] The other processing units basically perform the same processing as in the decoding device 200. However, the atlas information decoding unit 212 may supply the atlas information (including edge preservation information) to the initial edge list generation unit 414 and the display processing unit 220. The base mesh decoding unit 213 may supply the base mesh (including the triangle list) to the initial edge list generation unit 414. The subdivision unit 215 may acquire the initial edge list, base mesh, etc. supplied from the initial edge list generation unit 414.

[0162] With this configuration, the decoding device 400 can omit the search process and suppress an increase in the decoding load. Therefore, the decoding device 400 can suppress delays in the decoding process due to a lack of CPU resources. Furthermore, by acquiring edge preservation information, the decoding device 400 can suppress an increase in the code amount of the V-DMC bitstream compared to when the initial edge list is transmitted. Therefore, the decoding device 400 can suppress an increase in the transmission bandwidth usage. Furthermore, the decoding device 400 can suppress an increase in the decoding load due to an increase in the code amount of the V-DMC bitstream, and suppress delays in the decoding process due to a lack of CPU resources. In other words, the decoding device 400 can suppress an increase in the decoding load while suppressing an increase in the code amount.

[0163] <Flow of Decoding Process> An example of the flow of the decoding process executed by the decoding device 400 will be described with reference to the flowchart of FIG.

[0164] When the decoding process starts, the processes from step S401 to step S403 are executed in the same manner as the processes from step S201 to step S203 (FIG. 20), except that the atlas information obtained by decoding in step S402 includes edge preservation information.

[0165] In step S404, the initial edge list generation unit 414 executes an initial edge list generation process and generates an initial edge list using the edge preservation information.

[0166] When the process of step S404 is completed, the process proceeds to step S405. The processes of steps S405 to S410 are executed in the same manner as the processes of steps S204 to S209 (FIG. 20).

[0167] The decoding process ends when the process of step S410 is completed. The decoding device 400 executes such a decoding process for each frame of the original mesh.

[0168] <Flow of Initial Edge List Generation Processing> An example of the flow of the initial edge list generation processing executed in step S404 of FIG. 28 will be described with reference to the flowchart of FIG.

[0169] When the initial edge list generation process is started, in step S431, the initial edge list generation unit 414 executes ComputeNeighbors processing using edge preservation information to generate Neighbors that indicate vertices adjacent to each vertex without overlapping.

[0170] In step S432, the initial edge list generation unit 414 generates a non-overlapping edge list (i.e., an initial edge list) using the Neighbors. When the process of step S432 ends, the process returns to FIG.

[0171] By performing each process as described above, the decoding device 400 can omit the search process and suppress an increase in the decoding load. Therefore, the decoding device 400 can suppress delays in the decoding process due to a lack of CPU resources. Furthermore, by acquiring edge preservation information, the decoding device 400 can suppress an increase in the code amount of the V-DMC bitstream compared to when the initial edge list is transmitted. Therefore, the decoding device 400 can suppress an increase in the transmission bandwidth usage. Furthermore, the decoding device 400 can suppress an increase in the decoding load due to an increase in the code amount of the V-DMC bitstream, and suppress delays in the decoding process due to a lack of CPU resources. In other words, the decoding device 400 can suppress an increase in the decoding load while suppressing an increase in the code amount.

[0172] <Method 1-2> When Method 1 is applied, the edge preservation information may indicate the results of the overlap check for each edge in the overlapped edge list (Method 1-2), as shown in the third row from the top of the table in Fig. 21. In other words, the edge preservation information may correspond to an edge list that indicates, for each triangle, all combinations of vertices of all triangles in the base mesh.

[0173] In this case, the encoder may generate an edge list (edge ​​list with overlaps) for each triangle, check each edge element for overlaps, and generate a list of the check results as edge preservation information.

[0174] For example, in a third information processing device, the edge preservation information generation unit may generate an overlapping edge list that indicates the edges of the base mesh by all ``vertex combinations'' for each triangle, use the overlapping edge list to search for combinations that overlap with the combination to be processed, and generate edge preservation information using the results of the search.

[0175] Alternatively, the decoder may generate a similar overlapping edge list and use the overlapping edge list and edge preservation information to generate a unique edge list (i.e., the initial edge list).

[0176] For example, in a fourth information processing device, the initial edge list generation unit may generate an overlapping edge list that indicates the edges of the base mesh by all ``vertex combinations'' for each triangle, and generate an initial edge list using the overlapping edge list and edge preservation information.

[0177] That is, as shown in Figure 30, edge preservation information is supplied to the decoding process along with the base mesh. Then, in the process of generating the initial edge list, an overlapping edge list is generated, and a unique edge list is generated from the overlapping edge list. At this time, the edge preservation information is used. That is, the "search for combinations that overlap with the combination to be processed" is skipped for the overlapping edge list, and the edge preservation information is applied as the search result.

[0178] At that time, for example, overlapping edge elements may be deleted based on the edge preservation information, as shown in Fig. 31. For example, in the fourth information processing device, the initial edge list generation unit may generate the initial edge list by deleting overlapping "combinations of adjacent vertices of the base mesh" from the overlapping edge list based on the edge preservation information.

[0179] In the example of Figure 31, all vertex combinations are derived for each triangle in the triangle list, and an overlapping edge list (overlapping edge list) is generated with these combinations as elements. Each element of the edge preservation information corresponds to each element (combination) in this overlapping edge list. Therefore, combinations corresponding to elements whose edge preservation information value is true (e.g., "1") are stored in the non-overlapping edge list (unique list). In other words, combinations corresponding to elements whose edge preservation information value is false (e.g., "0") are not stored in the non-overlapping edge list (unique list) but are deleted.

[0180] In this way, it is possible to easily generate a non-duplicate edge list (i.e., an initial edge list) from an overlapping edge list, and it is possible to suppress an increase in the load of the decoding process.

[0181] Note that when generating a non-duplicate edge list from the overlapping edge list, the overlapping edge list may be sorted based on the edge preservation information and then duplicate combinations may be deleted. For example, as shown in Fig. 32 , the overlapping edge list may be sorted so as to group together combinations corresponding to elements whose edge preservation information values ​​are true (e.g., "1") (i.e., so as to classify combinations corresponding to elements whose edge preservation information values ​​are true (e.g., "1") from combinations corresponding to elements whose edge preservation information values ​​are false (e.g., "0")).

[0182] For example, in a fourth information processing device, the initial edge list generation unit may generate an initial edge list by sorting the overlapping edge list based on edge preservation information and deleting overlapping "combinations of adjacent vertices of the base mesh" from the sorted overlapping edge list.

[0183] In the example of FIG. 32 , the overlapping edge list is sorted so that combinations corresponding to elements whose edge preservation information value is true (for example, "1") come first. In other words, combinations to be deleted (combinations not registered in the overlapping edge list) are collected at the end. Therefore, the process of registering combinations in the overlapping edge list can be executed as a pipeline process in which combinations are registered in order starting from the first combination to be processed. Therefore, this process can be executed on a GPU. As a result, an increase in the CPU load can be suppressed.

[0184] <Encoding device and encoding process flow> In this method 1-2, the encoding device may have a configuration similar to that of the encoding device 300 (Figure 25). The encoding process may also be performed with a flow similar to that shown in the example flowchart of Figure 26. However, the edge preservation information generation unit 323 may (in step S303) generate an overlapping edge list that indicates the edges of the base mesh by all "vertex combinations" for each triangle, use the overlapping edge list to search for combinations that overlap with the combination to be processed, and generate edge preservation information using the search results. By doing so, the encoding device can achieve the effects described above.

[0185] <Decoding Device and Decoding Process Flow> In addition, in the case of Method 1-2, the decoding device may have a configuration similar to that of the decoding device 400 ( FIG. 27 ). Furthermore, the decoding process may be executed with a flow similar to that of the example flowchart of FIG. 28 . However, the initial edge list generation unit 414 may (in step S404) generate an overlapping edge list indicating the edges of the base mesh by all “vertex combinations” for each triangle, and generate the initial edge list using the overlapping edge list and the edge preservation information. In this case, the initial edge list generation unit 414 may generate the initial edge list by deleting overlapping “combinations of adjacent vertices of the base mesh” from the overlapping edge list based on the edge preservation information. Furthermore, the initial edge list generation unit 414 may sort the overlapping edge list based on the edge preservation information, and then generate the initial edge list by deleting overlapping “combinations of adjacent vertices of the base mesh” from the sorted overlapping edge list. In this way, the decoding device can achieve the above-described effects.

[0186] <Flow of Initial Edge List Generation Processing> An example of the flow of initial edge list generation processing when method 1-2 is applied will be described with reference to the flowchart in FIG.

[0187] When the initial edge list generation process starts, in step S451 the initial edge list generation unit 414 generates an overlapping edge list from the triangle list, as shown in Figures 31 and 32. For example, the initial edge list generation unit 414 may generate an overlapping edge list that indicates the edges of the base mesh by all "vertex combinations" for each triangle.

[0188] In step S452, the initial edge list generation unit 414 generates a non-overlapping edge list (i.e., an initial edge list) using the edge preservation information. For example, the initial edge list generation unit 414 may generate the initial edge list by deleting duplicated "combinations of adjacent vertices of the base mesh" from the duplicated edge list based on the edge preservation information. Alternatively, the initial edge list generation unit 414 may sort the duplicated edge list based on the edge preservation information and delete duplicated "combinations of adjacent vertices of the base mesh" from the sorted duplicated edge list to generate the initial edge list.

[0189] When the process of step S432 ends, the process returns to Fig. 28. By doing so, the decoding device can obtain the above-mentioned effects.

[0190] <Method 1-3> When Method 1 is applied, edge-by-edge edge preservation information may be transmitted (Method 1-3), as shown in the fourth row from the top of the table in FIG. 21 . That is, the edge preservation information may indicate, for each edge (for each "combination of adjacent vertices of the base mesh"), whether or not the edge is to be stored in the initial edge list. For example, the edge preservation information may be configured with flag information indicating, for each edge (each combination), whether or not the edge is to be stored in the initial edge list. In this case, the edge preservation information can be expressed with one bit per edge, thereby suppressing an increase in the amount of code. In this specification, such edge information is also referred to as edge-by-edge preservation information.

[0191] In this case, the encoder may generate edge-by-edge preservation information by setting the value of each flag information. For example, in the third information processing device, the edge preservation information generating unit may generate first edge preservation information (edge-by-edge preservation information) that indicates, as edge preservation information, whether to include, for each "combination of adjacent vertices of the base mesh," in the initial edge list.

[0192] In response to this, the decoder may control whether or not to include an edge in the initial edge list based on the value of each flag information in the edge-by-edge preservation information. For example, in the fourth information processing device, the edge preservation information (edge-by-edge preservation information) may include an edge element that indicates whether or not to include each "combination of adjacent vertices of the base mesh" in the initial edge list.

[0193] <Method 1-3-1> When Method 1-3 is applied, triangle-level edge storage information may also be transmitted (Method 1-3-1), as shown in the fifth row from the top of the table in Figure 21. As shown in Figure 34, the triangle-level edge storage information indicates, for each triangle of the base mesh, whether or not to store the edge in the initial edge list. In other words, the triangle-level edge storage information is composed of flag information that indicates, for each triangle, whether or not to store the edge (combination) in the initial edge list.

[0194] For example, in a third information processing device, the edge preservation information generation unit may further generate second edge preservation information (triangle-by-triangle edge preservation information) that indicates, for each triangle of the base mesh, whether to include it in the initial edge list as edge preservation information.

[0195] In addition, a triangle element whose value is true in the second edge preservation information (triangle-unit edge preservation information) may indicate that one or more of the "vertex combinations" of the triangle corresponding to that triangle element are to be included in the initial edge list.

[0196] Also, a triangle element whose value is true in the second edge preservation information (triangle-by-triangle edge preservation information) may indicate that all "combinations of vertices" of the triangle corresponding to that triangle element are to be included in the initial edge list.

[0197] The meaning of the value of this triangle element may be switched using control information, etc. For example, as described above, when a triangle element is true, it may be possible to switch between indicating that one or more of the "vertex combinations" of the triangle corresponding to the triangle element are to be included in the initial edge list, and indicating that all of the "vertex combinations" of the triangle corresponding to the triangle element are to be included in the initial edge list.

[0198] Furthermore, if not all edges of a triangle (all vertex combinations) are stored in the initial edge list, transmission of the edge-by-edge storage information for that triangle may be omitted. In other words, if all three edge elements of a triangle are false (e.g., "0"), the edge elements of that triangle may not be included (omitted) in the edge-by-edge storage information. In other words, the edge-by-edge storage information may be configured to include edge elements of a triangle in which at least one of the three edge elements is true. By omitting transmission of the edge-by-edge storage information in this way, an increase in the amount of code can be suppressed.

[0199] For example, if triangles are infinitely connected, arranging triangles that retain all edges adjacent to triangles that retain none allows edges to be retained without overlap. In this case, edge-by-edge storage information for approximately 50% of the triangles becomes unnecessary. Ideally, this would be 1 + 0.5 * 3 = 2.5 bits per triangle, reducing the amount of code by 0.5 bits per triangle compared to when only edge-by-edge storage information is used.

[0200] For example, suppose there is a mesh as shown in Figure 35. As shown in Figure 35, all edges of gray triangles are retained, but all edges of white triangles are not retained. In this case, it is possible to save bits compared to signaling only edge-by-edge information while retaining all edges without overlaps. For example, when only edge-by-edge information is used, the coding amount is 3 x 16 = 48 bits. In contrast, by using triangle-by-triangle information in addition, the coding amount becomes 1 x 16 + 3 x 10 = 46 bits. In other words, the coding amount can be reduced by 2 bits.

[0201] When such triangle-based edge storage information and edge-based edge storage information are transmitted, in the decoder, if both the corresponding triangle element of the triangle-based edge storage information and the corresponding edge element of the edge-based edge storage information are true, the edge may be stored in the initial edge list.

[0202] For example, in a fourth information processing device, the edge preservation information may further include a triangle element indicating, for each triangle of the base mesh, whether to include it in the initial edge list. Furthermore, the initial edge list generation unit may not include, in the initial edge list, a "vertex combination" of a triangle corresponding to a false triangle element. Furthermore, the initial edge list generation unit may not include, in the initial edge list, a "vertex combination" of a triangle corresponding to a true triangle element. Furthermore, the initial edge list generation unit may include, in the initial edge list, a "vertex combination" of a triangle corresponding to a true edge element.

[0203] 34, among the edge elements in the overlapping edge list, only edges that correspond to both triangle elements with true values ​​for the triangle-by-triangle edge storage information and edge elements with true values ​​for the edge-by-edge storage information are stored in the non-overlapping edge list (i.e., the initial edge list). In other words, all other edges are deleted.

[0204] In this way, a list of non-overlapping edges can be easily generated from a list of overlapping edges.

[0205] <Method 1-4> When Method 1 is applied, the edge preservation information may be coded and transmitted as shown in the sixth row from the top of the table in FIG. 21 (Method 1-4).

[0206] For example, in the third information processing apparatus, the atlas information encoding unit may perform run-length encoding on the edge preservation information. For example, the atlas information encoding unit 151 may perform run-length encoding on the edge preservation information.

[0207] In the fourth information processing device, the atlas information decoding unit may perform run-length decoding on the coded data of the edge preservation information. For example, the atlas information decoding unit 212 may perform run-length decoding on the coded data of the edge preservation information.

[0208] By doing so, it is possible to suppress an increase in the amount of data when transmitting edge preservation information, and it is also possible to suppress an increase in the amount of code in the V-DMC bitstream.

[0209] The edge preservation information may be coded in any manner, including coding methods other than run-length coding.

[0210] <Method 1-5> When Method 1 is applied, control information related to edge preservation information may be transmitted as shown in the seventh row from the top of the table in FIG. 21 (Method 1-5).

[0211] For example, the third information processing device may further include a control information generating unit that generates control information related to the edge preservation information. Also, the atlas information encoding unit may encode the atlas information including the edge preservation information and the control information to generate encoded data of the atlas information.

[0212] <Encoding device> Fig. 36 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which Method 1 (or Method 1-1) is applied. The encoding device 700 (third information processing device) shown in Fig. 36 is a device that encodes meshes, similar to the encoding device 300.

[0213] Fig. 36 shows the main processing units, data flows, etc., but does not necessarily include all of them. In other words, in encoding device 700, there may be processing units that are not shown as blocks in Fig. 36, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 36.

[0214] As shown in FIG. 36, the encoding device 700 (third information processing device) has the same configuration as the encoding device 300 , and further includes a control information generating unit 723 .

[0215] The control information generation unit 723 executes processing related to the generation of control information related to edge preservation information. For example, the control information generation unit 723 may generate control information related to edge preservation information. In this case, the control information generation unit 723 may acquire information necessary for generating the control information from the atlas information generation unit 122. The control information generation unit 723 may supply the generated control information to the atlas information generation unit 122 as atlas information.

[0216] The other processing units basically perform the same processing as in the encoding device 300. However, the edge preservation information generation unit 323 may acquire the control information and generate edge preservation information based on the control information. Furthermore, the atlas information encoding unit 151 may acquire the control information and encode the edge preservation information based on the control information. Furthermore, the atlas information encoding unit 151 may encode atlas information including the edge preservation information and the control information, and generate encoded data of the atlas information.

[0217] With this configuration, the encoding device 700 can provide control information for controlling the decoding process, and therefore the encoding device 700 can execute the decoding process more efficiently based on the control information, thereby suppressing an increase in the load of the decoding process.

[0218] <Flow of Encoding Process> An example of the flow of encoding process executed by this encoding device 700 will be described with reference to the flowchart of FIG.

[0219] When the encoding process is started, the processes of steps S701 and S702 are executed in the same manner as the processes of steps S301 and S302 (FIG. 26).

[0220] In step S703, the control information generator 723 generates control information related to edge preservation information. In step S704, the edge preservation information generator 323 generates edge preservation information based on the control information.

[0221] When the process of step S704 is completed, the process proceeds to step S705. The processes of steps S705 and S706 are executed in the same manner as the processes of steps S304 and S305 (FIG. 26). When the process of step S706 is completed, the encoding process ends. The encoding device 700 executes such encoding process for each frame of the original mesh.

[0222] The V-DMC encoding process executed in step S706 in FIG. 37 is executed in the same manner as in the example described with reference to the flowchart in FIG.

[0223] By performing each process as described above, the encoding device 700 can provide control information for controlling the decoding process, and therefore the encoding device 700 can perform the decoding process more efficiently based on the control information, thereby suppressing an increase in the load of the decoding process.

[0224] <Decoding Device> In the decoding device, the decoding process can be performed using the edge preservation information based on the control information.

[0225] For example, in the fourth information processing apparatus, the initial edge list generating unit may generate an initial edge list of the base mesh using control information relating to edge preservation information and edge preservation information included in the atlas information.

[0226] The decoding device in the case of method 1-5 may have a configuration similar to that of the decoding device 400 ( FIG. 27 ). However, the atlas information decoding unit 212 may supply control information and edge preservation information as atlas information to the initial edge list generation unit 414. The initial edge list generation unit 414 may generate an initial edge list using the control information and edge preservation information.

[0227] With this configuration, the decoding device 400 can more efficiently perform the decoding process based on the control information, and can suppress an increase in the load of the decoding process.

[0228] <Flow of Decoding Process> An example of the flow of the decoding process executed by the decoding device 400 when applying Method 1-5 will be described with reference to the flowchart in FIG.

[0229] When the decoding process starts, steps S801 to S803 are executed in the same manner as steps S401 to S403 (FIG. 28), except that the atlas information obtained by decoding in step S802 includes control information and edge preservation information.

[0230] In step S804, the initial edge list generation unit 414 executes an initial edge list generation process and generates an initial edge list using the control information and edge preservation information.

[0231] When the process of step S804 ends, the process proceeds to step S805. The processes of steps S805 to S810 are executed in the same manner as the processes of steps S405 to S410 (FIG. 28).

[0232] The decoding process ends when the process of step S810 is completed. The decoding device 400 executes such a decoding process for each frame of the original mesh.

[0233] By performing each process as described above, the decoding device 400 can perform the decoding process more efficiently based on the control information, and can suppress an increase in the load of the decoding process.

[0234] <Method 1-5-1> The content of the above-mentioned control information is arbitrary, and may include information for performing any control. For example, when Method 1-5 is applied, information regarding the validity / invalidity of edge preservation information may be transmitted as control information, as shown in the eighth row from the top of the table in Fig. 21 (Method 1-5-1). In other words, a validity flag indicating whether the edge preservation information is valid or invalid may be included in the above-mentioned control information.

[0235] The encoding side may set this valid flag, and further generate edge preserve information based on the valid flag. For example, in a third information processing device, a control information generation unit may generate, as control information, a valid flag indicating whether the edge preserve information is valid. Furthermore, the edge preserve information generation unit may generate edge preserve information when the valid flag is true. For example, in the encoding device 700, the control information generation unit 723 may set this valid flag. Furthermore, the edge preserve information generation unit 323 may generate edge preserve information when the valid flag is set to true. In other words, the edge preserve information generation unit 323 may skip (omit) generating edge preserve information when the valid flag is set to false.

[0236] The decoding side may acquire this validity flag, and further determine whether to use edge preservation information in generating an initial edge list based on the validity flag. For example, in a fourth information processing device, the control information may include a validity flag indicating whether the edge preservation information is valid. Furthermore, if the validity flag is true, the initial edge list generation unit may generate an initial edge list using the edge preservation information. Furthermore, if the validity flag is false, the initial edge list generation unit may generate an initial edge list without using the edge preservation information. For example, in the decoding device 400, the initial edge list generation unit 414 may acquire a validity flag supplied as atlas information. Then, if the validity flag is set to true, the initial edge list generation unit 414 may generate an initial edge list using the edge preservation information. That is, the initial edge list may be generated by applying, for example, Method 1-1 or Method 1-2. In other words, if the validity flag is set to false, the initial edge list generation unit 414 may generate an initial edge list without using the edge preservation information. That is, the initial edge list may be generated using the conventional method described with reference to FIG. 3.

[0237] By using the valid flag in this way, the decoding device can more easily determine whether or not edge preservation information is available.

[0238] <Method 1-5-2> For example, when Method 1-5 is applied, information regarding the type of edge preservation information may be transmitted as control information (Method 1-5-2), as shown in the ninth row from the top of the table in Fig. 21. In other words, information indicating the type of edge preservation information (method of generating the initial edge list) may be included in the above-mentioned control information.

[0239] This information may be set on the encoding side and provided to the decoding side. For example, in a third information processing device, a control information generation unit may generate, as control information, information indicating a method for generating an initial edge list. Furthermore, the edge preservation information generation unit may generate edge preservation information to be used in the indicated generation method. For example, in the encoding device 700, the control information generation unit 723 may set this "information indicating a method for generating an initial edge list." Furthermore, the edge preservation information generation unit 323 may generate edge preservation information to be used in the generation method indicated by the information.

[0240] The decoding side may acquire this "information indicating the method for generating the initial edge list" and generate the initial edge list by applying the generation method indicated by the information. For example, in a fourth information processing device, the control information may include information indicating the method for generating the initial edge list. Furthermore, the initial edge list generation unit may generate the initial edge list using the indicated generation method. For example, in the decoding device 400, the initial edge list generation unit 414 may generate the initial edge list using the indicated generation method. By doing so, it may be easier to select, for example, whether to apply Method 1-1 or Method 1-2.

[0241] <Method 1-5-3> For example, when Method 1-5 is applied, information regarding the unit of edge preservation information may be transmitted as control information (Method 1-5-3), as shown in the tenth row from the top of the table in Fig. 21. For example, the control information may include information indicating whether the edge preservation information is composed of only edge-unit edge preservation information or also includes triangle-unit edge preservation information.

[0242] For example, in the third information processing device, the control information generation unit may generate, as the control information, information indicating a unit of edge preservation information. Also, the edge preservation information generation unit may generate edge preservation information having the indicated unit as an element.

[0243] For example, in the fourth information processing device, the control information may include information indicating a unit of edge preservation information, and the initial edge list generation unit may generate the initial edge list using elements of the edge preservation information in the indicated unit.

[0244] By doing so, the unit of edge preservation information can be controlled more easily.

[0245] <Method 1-5-4> When Method 1-5 is applied, information regarding the encoding of edge-preserving information may be transmitted as control information (Method 1-5-4), as shown in the bottom row of the table in Fig. 21. For example, the control information may include an encoding flag indicating whether or not to encode the edge-preserving information.

[0246] For example, in the third information processing device, the control information generation unit may generate, as control information, an encoding flag indicating whether to encode the edge preservation information, and the atlas information encoding unit may encode the edge preservation information when the encoding flag is true.

[0247] For example, in a fourth information processing device, the control information may include an encoding flag indicating whether to encode the edge preservation information. Furthermore, the atlas information decoding unit may decode encoded data of the edge preservation information if the encoding flag is true. Furthermore, the atlas information decoding unit may skip decoding the edge preservation information if the encoding flag is false.

[0248] By using such an encoding flag, it is possible to more easily control whether or not to encode edge preservation information.

[0249] Furthermore, information indicating a coding method for the edge preservation information may be used as the control information. For example, in the third information processing device, the control information generation unit may generate, as the control information, information indicating a coding method for the edge preservation information. Furthermore, the atlas information encoding unit may encode the edge preservation information using the indicated coding method.

[0250] For example, in a fourth information processing device, the control information may include information indicating an encoding method for the edge preservation information, and the atlas information decoding unit may decode the encoded data of the edge preservation information using a decoding method corresponding to the indicated encoding method.

[0251] In this way, it is easier to control how the edge preserving information is coded.

[0252] <Combination> Each of the above-described methods may be applied in combination with any other method as long as no contradiction occurs. Three or more methods may be applied in combination. Furthermore, the methods that can be combined are not limited to those shown in the table of FIG. 10 as "methods," but may include all of the elements described above in <3. Transmission of initial edge list> and <4. Transmission of edge preservation information>. Furthermore, each of the above-described methods may be applied in combination with methods other than those described above.

[0253] In this specification, a description of a higher-level method also applies to lower-level methods that belong to that higher-level method, unless a contradiction arises. For example, when it is described that "Method 1 may be applied," each of Methods 1-1 to 1-5 can be applied. Furthermore, when it is described that "Method 1 may be applied," each of Methods 1-1 to 1-5 can be applied, and Method 1-3-1 and each of Methods 1-5-1 to 1-5-4 can also be applied.

[0254] <Example of Pseudo Program> Figures 39 to 42 are diagrams showing an example of a pseudo program for the process of deriving an initial edge list. Figure 40 is a diagram showing an example of a pseudo program for the ComputeNeighbors process of Figure 39. Figure 41 is a diagram showing an example of a pseudo program for the InitEdgeListGpufriendly process of Figure 39. Figure 42 is a diagram showing an example of a pseudo program for the reconEdgePreserveInfo process of Figure 39. As shown in this pseudo program, the process of deriving an initial edge list can be easily controlled based on control information.

[0255] 5. Supplementary Notes Polygon Shape In the above description, the polygon shape is described as being triangular, but this shape is merely an example. The polygon shape may be any polygonal shape.

[0256] <Scope of Application of the Present Technology> In the above, V-DMC has been used as an example of an encoding method to which the present technology can be applied, but the present technology is not limited to this example, and can be applied to any encoding method that encodes a base mesh, a displacement vector, an attribute map including texture, and atlas information, or information equivalent thereto.

[0257] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0258] FIG. 43 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0259] In a computer 900 shown in FIG. 43, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0260] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0261] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 912 includes, for example, a display, a speaker, and an output terminal. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0262] In a computer configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0263] The program executed by the computer can be applied by recording it on, for example, a removable medium 921 such as a package medium. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by inserting the removable medium 921 into the drive 915.

[0264] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0265] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0266] <Application of the Present Technology> The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0267] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set in which other functions are added to a unit (e.g., a video set).

[0268] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0269] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0270] <Fields and uses to which this technology can be applied> Systems, devices, processing units, etc. to which this technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, the uses thereof are also arbitrary.

[0271] <Others> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in a bitstream, but also in a bitstream that includes differential information of the identification information relative to certain reference information. Therefore, in this specification, "flag" and "identification information" encompass not only the information itself, but also differential information relative to the reference information.

[0272] Furthermore, various types of information (e.g., metadata) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, the associated pieces of data may be combined into one piece of data or may be separate pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Furthermore, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer not to the entire data, but to only part of the data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0273] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0274] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0275] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0276] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0277] Also, for example, each step of a single flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, multiple processes included in a single step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0278] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0279] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0280] The present technology can also be configured as follows. (1) An edge preservation information generation unit that generates edge preservation information indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list; an atlas information encoding unit that encodes atlas information including the edge preservation information and generates encoded data of the atlas information; and a bitstream generation unit that generates a bitstream including the encoded data of the base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, the displacement video is a video in which frame images are a displacement map, which is a two-dimensional region in which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; and the attribute video is a video in which frame images are an attribute map, which is a two-dimensional region in which the texture of the original mesh is packed; and the atlas information is information used for mesh reconstruction, including information indicating a correspondence between the base mesh, the displacement map, and the attribute map. The information processing device according to (1), wherein the initial edge list is a unique edge list that indicates the edges of the base mesh by the non-overlapping combinations. (2) The information processing device according to (1), wherein the edge preservation information generation unit is configured to execute ComputeNeighbors processing on the base mesh and generate the edge preservation information using a result of searching for the overlapping combinations. (3) The information processing device according to (1), wherein the edge preservation information generation unit is configured to generate an overlapping edge list that indicates the edges of the base mesh by all the combinations for each triangle, search for the overlapping combinations using the overlapping edge list, and generate the edge preservation information using a result of the search.(4) The information processing device according to any one of (1) to (3), wherein the edge preservation information generation unit is configured to generate first edge preservation information indicating, for each combination, whether to include the combination in the initial edge list as the edge preservation information. (5) The information processing device according to (4), wherein the edge preservation information generation unit is further configured to generate second edge preservation information indicating, for each triangle of the base mesh, whether to include the combination in the initial edge list as the edge preservation information. (6) The information processing device according to (5), wherein a triangle element having a value of true in the second edge preservation information indicates that one or more of the combinations of the triangles corresponding to the triangle element are to be included in the initial edge list. (7) The information processing device according to (5), wherein a triangle element having a value of true in the second edge preservation information indicates that all the combinations of the triangles corresponding to the triangle element are to be included in the initial edge list. (8) The information processing device according to any one of (1) to (7), wherein the atlas information encoding unit is configured to perform run-length encoding of the edge preservation information. (9) The information processing device according to any of (1) to (8), further comprising a control information generation unit that generates control information related to the edge preservation information, wherein the atlas information encoding unit encodes the atlas information including the edge preservation information and the control information to generate encoded data of the atlas information. (10) The information processing device according to (9), wherein the control information generation unit is configured to generate, as the control information, a validity flag indicating whether the edge preservation information is valid, and the edge preservation information generation unit is configured to generate the edge preservation information when the validity flag is true. (11) The information processing device according to (9) or (10), wherein the control information generation unit is configured to generate, as the control information, information indicating a method for generating the initial edge list, and the edge preservation information generation unit is configured to generate the edge preservation information used in the indicated generation method.(12) The information processing device according to any of (9) to (11), wherein the control information generation unit is configured to generate, as the control information, information indicating a unit of the edge preservation information, and the edge preservation information generation unit is configured to generate the edge preservation information having the indicated unit as an element. (13) The information processing device according to any of (9) to (12), wherein the control information generation unit is configured to generate, as the control information, an encoding flag indicating whether the edge preservation information is to be encoded, and the atlas information encoding unit is configured to encode the edge preservation information if the encoding flag is true. (14) The information processing device according to any of (9) to (13), wherein the control information generation unit is configured to generate, as the control information, information indicating an encoding method of the edge preservation information, and the atlas information encoding unit is configured to encode the edge preservation information using the indicated encoding method.(15) Generate edge preservation information indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list; encode atlas information including the edge preservation information to generate encoded data of the atlas information; generate a bitstream including encoded data of the base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information; the base mesh is a mesh with lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement video is a video having frame images of a displacement map, which is a two-dimensional area in which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; the attribute video is a video having frame images of an attribute map, which is a two-dimensional area in which the texture of the original mesh is packed; and the atlas information is information used for mesh reconstruction, including information indicating a correspondence between the base mesh, the displacement map, and the attribute map; The information processing method, wherein the initial edge list is a unique edge list that indicates the edges of the base mesh by the non-overlapping combinations.

[0281] (21) A video encoding system comprising: a demultiplexing unit that demultiplexes a bitstream and extracts coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information; an atlas information decoding unit that decodes the extracted coded data of the atlas information; a base mesh decoding unit that decodes the extracted coded data of the base mesh; an initial edge list generating unit that generates an initial edge list of the base mesh using edge preservation information included in the atlas information that indicates whether each combination of adjacent vertices of the base mesh should be included as an edge in an initial edge list; and a subdivision unit that subdivides the base mesh using the initial edge list, wherein the base mesh is a mesh with lower resolution than an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object and is generated by thinning out vertices from the original mesh, and the displacement video is a moving image whose frame images are displacement maps that are two-dimensional regions in which displacement vectors that indicate displacements of vertices of the subdivided base mesh are packed, The information processing device according to (21), wherein the attribute video is a moving image having frame images of an attribute map, which is a two-dimensional region in which the texture of the original mesh is packed, the atlas information is information used for reconstructing a mesh, including information indicating a correspondence between the base mesh, the displacement map, and the attribute map, and the initial edge list is a unique edge list indicating the edges of the base mesh by the non-overlapping combinations. (22) The information processing device according to (21), wherein the initial edge list generation unit is configured to execute ComputeNeighbors processing using the edge preservation information to generate Neighbors indicating vertices adjacent to each vertex without overlapping, and to generate the initial edge list using the Neighbors.(23) The information processing device according to (21), wherein the initial edge list generation unit is configured to generate a duplicated edge list indicating all the combinations of the edges of the base mesh for each triangle, and to generate the initial edge list using the duplicated edge list and the edge preservation information. (24) The information processing device according to (23), wherein the initial edge list generation unit is configured to generate the initial edge list by deleting duplicated combinations from the duplicated edge list based on the edge preservation information. (25) The information processing device according to (23), wherein the initial edge list generation unit is configured to sort the duplicated edge list based on the edge preservation information and delete duplicated combinations from the sorted duplicated edge list, to generate the initial edge list. (26) The information processing device according to any of (21) to (25), wherein the edge preservation information includes an edge element that indicates, for each combination, whether to include it in the initial edge list. (27) The information processing device according to (26), wherein the edge preservation information further includes a triangle element indicating, for each triangle of the base mesh, whether to include it in the initial edge list, and the initial edge list generation unit is configured to: not include in the initial edge list the combination of the triangles corresponding to the false triangle elements, not include in the initial edge list the combination of the triangles corresponding to the true triangle elements and the false edge elements, and include in the initial edge list the combination of the triangles corresponding to the true triangle elements and the true edge elements. (28) The information processing device according to any of (21) to (27), wherein the atlas information decoding unit is configured to run-length decode encoded data of the edge preservation information. (29) The information processing device according to any of (21) to (28), wherein the initial edge list generation unit is configured to generate the initial edge list of the base mesh using control information related to the edge preservation information and the edge preservation information, which are included in the atlas information.(30) The information processing device according to (29), wherein the control information includes a validity flag indicating whether the edge preservation information is valid, and the initial edge list generation unit is configured to generate the initial edge list using the edge preservation information if the validity flag is true, and to generate the initial edge list without using the edge preservation information if the validity flag is false. (31) The information processing device according to (29) or (30), wherein the control information includes information indicating a generation method of the initial edge list, and the initial edge list generation unit is configured to generate the initial edge list using the indicated generation method. (32) The information processing device according to any of (29) to (31), wherein the control information includes information indicating a unit of the edge preservation information, and the initial edge list generation unit is configured to generate the initial edge list using elements of the edge preservation information of the indicated unit. (33) The information processing device according to any of (29) to (32), wherein the control information includes an encoding flag indicating whether to encode the edge preserve information, and the atlas information decoding unit is configured to: decode encoded data of the edge preserve information when the encoding flag is true, and skip decoding of the edge preserve information when the encoding flag is false. (34) The information processing device according to any of (29) to (33), wherein the control information includes information indicating an encoding method of the edge preserve information, and the atlas information decoding unit is configured to decode encoded data of the edge preserve information by a decoding method corresponding to the indicated encoding method.(35) Demultiplexing a bitstream to extract coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information; decoding the extracted coded data of the atlas information; decoding the extracted coded data of the base mesh; generating the initial edge list of the base mesh using edge preservation information included in the atlas information indicating whether each combination of adjacent vertices of the base mesh should be included as an edge in an initial edge list; subdividing the base mesh using the initial edge list; the base mesh is a mesh with lower resolution than an original mesh to be coded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement video is a moving image having frame images of a displacement map, which is a two-dimensional area in which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; and the attribute video is a moving image having frame images of an attribute map, which is a two-dimensional area in which the texture of the original mesh is packed. An information processing method, in which the atlas information is information used to reconstruct a mesh, including information indicating the correspondence between the base mesh and the displacement map and the attribute map, and the initial edge list is a unique edge list that indicates the edges of the base mesh by the non-overlapping combinations.

[0282] (41) A video encoding system comprising: an initial edge list generation unit that generates an initial edge list, which is a unique edge list indicating edges of a base mesh by non-overlapping combinations of adjacent vertices of the base mesh; an atlas information encoding unit that encodes atlas information including the initial edge list and generates encoded data of the atlas information; and a bitstream generation unit that generates a bitstream including the encoded data of the base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information, wherein the base mesh is a mesh with lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement video is a video having frame images of a displacement map, which is a two-dimensional region in which displacement vectors indicating displacements of vertices of the subdivided base mesh are packed; and the attribute video is a video having frame images of an attribute map, which is a two-dimensional region in which the texture of the original mesh is packed. The atlas information is information used for reconstructing a mesh, and includes information indicating a correspondence between the base mesh, the displacement map, and the attribute map.(42) An information processing method comprising: generating an initial edge list, which is a unique edge list indicating edges of a base mesh by non-overlapping combinations of adjacent vertices of the base mesh; encoding atlas information including the initial edge list to generate encoded data of the atlas information; generating a bitstream including encoded data of the base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information; wherein the base mesh is a mesh with lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement video is a moving image having frame images of a displacement map, which is a two-dimensional region in which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; the attribute video is a moving image having frame images of an attribute map, which is a two-dimensional region in which the texture of the original mesh is packed; and the atlas information is information used for mesh reconstruction, including information indicating a correspondence between the base mesh, the displacement map, and the attribute map.

[0283] (51) A video encoding system comprising: a demultiplexing unit that demultiplexes a bitstream and extracts coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information; an atlas information decoding unit that decodes the extracted coded data of the atlas information; a base mesh decoding unit that decodes the extracted coded data of the base mesh; and a subdivision unit that subdivides the base mesh using an initial edge list included in the atlas information, which is a unique edge list that indicates edges of the base mesh by non-overlapping combinations of adjacent vertices of the base mesh; wherein the base mesh is a mesh with lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement video is a video having frame images of a displacement map that is a two-dimensional area in which displacement vectors indicating displacements of vertices of the subdivided base mesh are packed; and the attribute video is a video having frame images of an attribute map that is a two-dimensional area in which textures of the original mesh are packed. The atlas information is information used for reconstructing a mesh, and includes information indicating a correspondence between the base mesh, the displacement map, and the attribute map.(52) Demultiplexing a bitstream to extract coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information; decoding the extracted coded data of the atlas information; decoding the extracted coded data of the base mesh; subdividing the base mesh using an initial edge list included in the atlas information, which is a unique edge list indicating edges of the base mesh by non-overlapping combinations of adjacent vertices of the base mesh; the base mesh is a mesh with lower resolution than an original mesh to be coded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement video is a video having frame images of a displacement map, which is a two-dimensional area in which displacement vectors indicating displacements of vertices of the subdivided base mesh are packed; and the attribute video is a video having frame images of an attribute map, which is a two-dimensional area in which textures of the original mesh are packed. The atlas information is information used for reconstructing a mesh, and includes information indicating a correspondence between the base mesh, the displacement map, and the attribute map.

[0284] 100 Encoding device, 111 Preprocessing unit, 112 V-DMC encoding unit, 121 Mesh decimation unit, 122 Atlas information generation unit, 123 Initial edge list generation unit, 124 Displacement vector generation unit, 151 Atlas information encoding unit, 152 Base mesh encoding unit, 153 Displacement vector correction unit, 154 Displacement video encoding unit, 155 Mesh reconstruction unit, 156 Attribute map conversion unit, 157 Attribute video encoding unit, 158 Multiplexing unit, 200 Decoding device, 211 Demultiplexing unit, 212 Atlas information decoding unit, 213 Base mesh decoding unit, 215 Subdivision unit, 216 Displacement video decoding unit, 217 Unpacking unit, 218 Displacement vector application unit, 219 Attribute video decoding unit, 220 Display processing unit, 300 encoding device, 323 edge preservation information generation unit, 400 decoding device, 414 initial edge list generation unit, 700 encoding device, 723 control information generation unit, 900 computer

Claims

1. An edge preservation information generation unit that generates edge preservation information indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list; an atlas information encoding unit that encodes atlas information including the edge preservation information and generates encoded data of the atlas information; and a bitstream generation unit that generates a bitstream including the encoded data of the base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, the displacement video is a video in which frame images are a displacement map, which is a two-dimensional area in which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; and the attribute video is a video in which frame images are an attribute map, which is a two-dimensional area in which the texture of the original mesh is packed; and the atlas information is information used for mesh reconstruction, including information indicating the correspondence between the base mesh, the displacement map, and the attribute map, The information processing device, wherein the initial edge list is a unique edge list that indicates the edges of the base mesh by the non-overlapping combinations.

2. The information processing device according to claim 1, wherein the edge preservation information generation unit is configured to execute ComputeNeighbors processing on the base mesh and generate the edge preservation information using a result of searching for overlapping combinations.

3. The information processing device according to claim 1, wherein the edge preservation information generation unit is configured to generate an overlapping edge list indicating the edges of the base mesh by all the combinations for each triangle, search for overlapping combinations using the overlapping edge list, and generate the edge preservation information using the search results.

4. The information processing device according to claim 1, wherein the edge preservation information generating unit is configured to generate, as the edge preservation information, first edge preservation information indicating, for each of the combinations, whether to include the combination in the initial edge list.

5. The information processing device according to claim 4, wherein the edge preservation information generation unit is configured to further generate second edge preservation information indicating, for each triangle of the base mesh, whether to include it in the initial edge list as the edge preservation information.

6. The information processing device according to claim 1, wherein the atlas information encoding unit is configured to perform run-length encoding of the edge preservation information.

7. An information processing device as described in claim 1, further comprising a control information generation unit that generates control information related to the edge preservation information, and wherein the atlas information encoding unit encodes the atlas information including the edge preservation information and the control information, and generates encoded data of the atlas information.

8. The information processing device according to claim 7, wherein the control information generation unit is configured to generate, as the control information, a validity flag indicating whether the edge preservation information is valid, and the edge preservation information generation unit is configured to generate the edge preservation information when the validity flag is true.

9. The information processing device according to claim 7, wherein the control information generation unit is configured to generate, as the control information, information indicating a method for generating the initial edge list, and the edge preservation information generation unit is configured to generate the edge preservation information to be used in the indicated generation method.

10. Generate edge preservation information indicating whether each combination of adjacent vertices of a base mesh should be included as an edge in an initial edge list; encode atlas information including the edge preservation information and generate encoded data of the atlas information; generate a bitstream including encoded data of the base mesh, encoded data of a displacement video, encoded data of an attribute video, and encoded data of the atlas information; the base mesh is a mesh with lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement video is a video in which frame images are a displacement map, which is a two-dimensional area in which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; the attribute video is a video in which frame images are an attribute map, which is a two-dimensional area in which the texture of the original mesh is packed; the atlas information is information used for mesh reconstruction, including information indicating the correspondence between the base mesh, the displacement map, and the attribute map; The information processing method, wherein the initial edge list is a unique edge list that indicates the edges of the base mesh by the non-overlapping combinations.

11. A video encoding system comprising: a demultiplexing unit that demultiplexes a bitstream and extracts coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information; an atlas information decoding unit that decodes the extracted coded data of the atlas information; a base mesh decoding unit that decodes the extracted coded data of the base mesh; an initial edge list generating unit that generates an initial edge list of the base mesh using edge preservation information included in the atlas information that indicates whether each combination of adjacent vertices of the base mesh should be included as an edge in an initial edge list; and a subdivision unit that subdivides the base mesh using the initial edge list, wherein the base mesh is a mesh with lower resolution than an original mesh to be encoded that is composed of vertices and connections that represent the three-dimensional structure of an object and is generated by thinning out vertices from the original mesh, and the displacement video is a video having frame images that are displacement maps that are two-dimensional regions in which displacement vectors that indicate the displacement of vertices of the subdivided base mesh are packed, An information processing device, wherein the attribute video is a moving image in which an attribute map, which is a two-dimensional area in which the texture of the original mesh is packed, is used as a frame image; the atlas information is information used to reconstruct a mesh, including information indicating the correspondence between the base mesh, the displacement map, and the attribute map; and the initial edge list is a unique edge list indicating the edges of the base mesh by the non-overlapping combinations.

12. The information processing device according to claim 11, wherein the initial edge list generation unit is configured to execute ComputeNeighbors processing using the edge preservation information to generate Neighbors indicating vertices adjacent to each vertex without overlapping, and to generate the initial edge list using the Neighbors.

13. The information processing device according to claim 11, wherein the initial edge list generation unit is configured to generate an overlapping edge list indicating the edges of the base mesh by all the combinations for each triangle, and to generate the initial edge list using the overlapping edge list and the edge preservation information.

14. The information processing device according to claim 13, wherein the initial edge list generation unit is configured to generate the initial edge list by deleting overlapping combinations from the overlapping edge list based on the edge preservation information.

15. The information processing device according to claim 13, wherein the initial edge list generation unit is configured to generate the initial edge list by sorting the overlapping edge list based on the edge preservation information and deleting the overlapping combinations from the sorted overlapping edge list.

16. The information processing device according to claim 11, wherein the edge preservation information includes an edge element that indicates whether each of the combinations is to be included in the initial edge list.

17. The information processing device of claim 16, wherein the edge preservation information further includes a triangle element indicating for each triangle of the base mesh whether to include it in the initial edge list, and the initial edge list generation unit is configured to: not include in the initial edge list the combination of triangles corresponding to false triangle elements; not include in the initial edge list the combination of triangles corresponding to true triangle elements corresponding to false edge elements; and include in the initial edge list the combination of triangles corresponding to true triangle elements corresponding to true edge elements.

18. The information processing device according to claim 11, wherein the atlas information decoding unit is configured to perform run-length decoding on the encoded data of the edge preservation information.

19. The information processing device according to claim 11, wherein the initial edge list generation unit is configured to generate the initial edge list of the base mesh using control information relating to the edge preservation information and the edge preservation information included in the atlas information.

20. Demultiplexing a bitstream to extract coded data of a base mesh, coded data of a displacement video, coded data of an attribute video, and coded data of atlas information; decoding the extracted coded data of the atlas information; decoding the extracted coded data of the base mesh; generating the initial edge list of the base mesh using edge preservation information included in the atlas information indicating whether each combination of adjacent vertices of the base mesh should be included as an edge in an initial edge list; subdividing the base mesh using the initial edge list; the base mesh is a mesh with lower resolution than an original mesh to be coded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement video is a video having frame images of a displacement map, which is a two-dimensional area in which displacement vectors indicating the displacement of vertices of the subdivided base mesh are packed; and the attribute video is a video having frame images of an attribute map, which is a two-dimensional area in which the texture of the original mesh is packed. An information processing method, in which the atlas information is information used to reconstruct a mesh, including information indicating the correspondence between the base mesh and the displacement map and the attribute map, and the initial edge list is a unique edge list that indicates the edges of the base mesh by the non-overlapping combinations.

Images