Information processing device and method

Abstract

The present disclosure relates to an information processing device and method with which it is possible to suppress any reduction in subjective quality and to suppress any increase in processing load related to mesh reconfiguration. Priority control information, which is information relating to the priority of a process related to mesh reconfiguration, is generated, and a base mesh, a displacement vector, and the priority control information pertaining thereto are encoded to generate a bit stream. Additionally, a bit stream of the priority control information that is the information relating to the priority of a process related to mesh reconfiguration is decoded, the base mesh is subdivided as a process related to the reconfiguration, and the displacement vector is applied to the subdivided mesh in accordance with a priority order based on the priority control information as a process related to the reconfiguration. The present disclosure can be applied to, e.g., information processing devices, electronic apparatuses, information processing methods, and programs.

Description

Information processing device and method

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that can suppress a decrease in subjective quality and suppress an increase in the processing load related to mesh reconstruction.

[0002] Conventionally, V-DMC (Video-based Dynamic Mesh Coding) has been used as a method for encoding meshes, which are 3D data that represent the three-dimensional structure of an object using vertices and connections (see, for example, Non-Patent Document 1). In V-DMC, a base mesh, displacement vectors for vertex correction, attributes, and atlas information are encoded and transmitted as a bitstream. During decoding, each piece of information in the bitstream is decoded, the base mesh is subdivided, and displacement vectors are applied to the subdivided vertices to reconstruct a decoded mesh. Generally, the greater the amount of information in a decoded mesh, the greater the processing load related to the reconstruction of the decoded mesh. However, the smaller the amount of information in the decoded mesh, the lower the subjective quality of the decoded mesh. In other words, in order to suppress the increase in the processing load related to the reconstruction of the mesh while suppressing the reduction in subjective quality, it is necessary to reduce the amount of information in the decoded mesh that has less impact on the subjective quality.

[0003] Khaled Mammou, Jungsun Kim, Alexis Tourapis, Dimitri Podborski, Krasimir Kolarov, "[V-CG] Apple's Dynamic Mesh Coding CfP Response", ISO / IEC JTC 1 / SC 29 / WG 7 m59281, April 2022

[0004] However, conventional methods have not provided a way for a decoding device to control which mesh region is given priority for reconstruction, making it difficult to suppress a decrease in subjective quality while suppressing an increase in the processing load related to mesh reconstruction.

[0005] The present disclosure has been made in view of such circumstances, and makes it possible to suppress a decrease in subjective quality and to suppress an increase in the processing load related to mesh reconstruction.

[0006] An information processing device according to one aspect of the present technology includes a priority control information decoding unit that decodes a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction; a subdivision unit that subdivides a base mesh as part of the processing related to the reconstruction; and a displacement vector application unit that applies a displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information as part of the processing related to the reconstruction, wherein the base mesh is a mesh with lower resolution than the original mesh that is generated by thinning vertices from an original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and the displacement vector is vector information that indicates the displacement of the vertices of the subdivided base mesh.

[0007] An information processing method according to one aspect of the present technology includes decoding a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction; subdividing a base mesh as processing related to the reconstruction; and applying a displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information as processing related to the reconstruction, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning vertices from the original mesh; and the displacement vector is vector information indicating the displacement of the vertices of the subdivided base mesh.

[0008] Another aspect of the present technology is an information processing device that includes a priority control information generation unit that generates priority control information, which is information regarding the priority of processing related to mesh reconstruction, and an encoding unit that encodes a base mesh, a displacement vector, and the priority control information and generates a bitstream, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, and the displacement vector is vector information that indicates the displacement of the vertices of the subdivided base mesh.

[0009] Another aspect of the information processing method of the present technology includes generating priority control information, which is information regarding processing priorities for mesh reconstruction, and encoding a base mesh, a displacement vector, and the priority control information to generate a bitstream, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, and the displacement vector is vector information indicating the displacement of the vertices of the subdivided base mesh.

[0010] In one aspect of the information processing device and method of the present technology, a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction, is decoded, and as part of the processing related to the reconstruction, a base mesh is subdivided, and as part of the processing related to the reconstruction, a displacement vector is applied to the subdivided base mesh in a priority order based on the priority control information.

[0011] Another aspect of the information processing device and method of the present technology includes generating priority control information, which is information regarding the processing priority for mesh reconstruction, and encoding the base mesh, displacement vector, and the priority control information to generate a bitstream.

[0012] 21 。 FIG. 22 is a diagram for explaining meshes. FIG. 23 is a diagram for explaining V-DMC. FIG. 24 is a diagram for explaining the relationship between the number of faces and load. FIG. 25 is a diagram for explaining an example of a method for controlling processing related to mesh reconstruction. FIG. 26 is a diagram for explaining an example of a method for controlling processing related to mesh reconstruction. FIG. 27 is a diagram for explaining an example of a method for controlling processing related to mesh reconstruction. FIG. 28 is a diagram for explaining an example of a mesh range specification. FIG. 29 is a diagram for explaining an example of a mesh range specification. FIG. 29 is a diagram for explaining an example of a displacement range specification. FIG. 29 is a diagram for explaining an example of a method for controlling processing related to mesh reconstruction. FIG. 29 is a diagram for explaining an example of a mesh range specification using texture information. FIG. 29 is a diagram for explaining an example of a mesh range specification using texture information. A block diagram showing an example of the main configuration of an encoding device. A block diagram showing an example of the main configuration of a V-DMC encoding unit. A flowchart for explaining an example of the flow of encoding processing. A flowchart for explaining an example of the flow of V-DMC encoding processing. A block diagram showing an example of the main configuration of a decoding device. A flowchart for explaining an example of the flow of decoding processing. A flowchart for explaining an example of the flow of decoding mesh reconstruction processing. A flowchart continuing from FIG. 21 , for explaining an example of the flow of decoding mesh reconstruction processing. A flowchart for explaining an example of the flow of decoding mesh reconstruction processing. 24 is a flowchart continuing from FIG. 23 , illustrating an example of the flow of a decoding mesh reconstruction process. FIG. 25 is a block diagram illustrating an example of the main configuration of an encoding device. FIG. 26 is a block diagram illustrating an example of the main configuration of a V-DMC encoding unit. FIG. 27 is a flowchart illustrating an example of the flow of encoding process. FIG. 28 is a flowchart illustrating an example of the flow of V-DMC encoding process. FIG. 29 is a block diagram illustrating an example of the main configuration of a decoding device. FIG. 29 is a flowchart illustrating an example of the flow of decoding process. FIG. 29 is a block diagram illustrating an example of the main configuration of a computer.

[0013] Modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below. The description will be made in the following order: 1. Literature, etc. supporting technical content and technical terminology 2. Decoding of V-DMC bitstream and mesh reconstruction 3. Priority control of processing related to mesh reconstruction 4. Local control of processing related to reconstruction using texture information 5. Priority control of processing related to reconstruction using texture information 6. First embodiment (encoding device) 7. Second embodiment (decoding device) 8. Third embodiment (encoding device) 9. Fourth embodiment (decoding device) 10. Supplementary notes

[0014] <1. Literature, etc. supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the content described in the embodiments, but also the content described in the following non-patent documents, etc. that were publicly known at the time of filing, and the content of other documents referenced in the following non-patent documents.

[0015] Non-patented Document 1: (above) Non-patented Document 2: "Information technology . Coded representation of immersive media . Part 29: Video-based dynamic mesh coding (V-DMC)", ISO 23090-29:2024(E), ISO / IEC JTC 1 / SC 29 / WG 7, MDS23617 WG07 N00822, 2024-03-15 Non-patented Document 3: Alexis Tourapis, Jungsun Kim, Dimitri Podborski, Khaled Mammou, "Base mesh data substream format for VDMC", ISO / IEC JTC 1 / SC 29 / WG 7 m60362, July 2022 Non-patented Document 4: Jungsun Kim, Alexis Tourapis, Dimitri Podborski, Khaled Mammou, David Flynn, "VDMC support in the V3C framework", ISO / IEC JTC 1 / SC 29 / WG 7 m60363, July 2022 Unlicensed Document 5: Danillo Bracco Graziosi, Alexandre Zaghetto, Ali Tabatabai, "[V-DMC][EE4.11] Zippering method for sub-meshes integration report", ISO / IEC JTC 1 / SC 29 / WG 7 m64768, October 2023 Unauthorized Document 6: Wenjie Zou, Chengcheng Li, Fuzheng Yang, Zhuoyi Lv, "[V-DMC][new] A segment-based base mesh inter-frame coding scheme", ISO / IEC JTC 1 / SC 29 / WG 7 m66383, January 2024

[0016] In other words, the contents of the above-mentioned non-patent documents and the contents of other documents referenced in the above-mentioned non-patent documents are also used as the basis for determining the support requirements.

[0017] <2. Decoding V-DMC bitstream and reconstructing mesh> <V-DMC> Conventionally, 3D data representing the three-dimensional structure of a three-dimensional structure (object with a three-dimensional shape) has been available as a mesh, which represents the three-dimensional shape of the object surface by forming polygons with vertices and connections (also called edges).

[0018] As shown in the upper left of Figure 1, in a mesh, vertices 11 and connections 12 connecting these vertices 11 form polygonal planes (polygons). These polygons (also called faces) represent the surface of a three-dimensional object, i.e., the three-dimensional shape of the object. A texture 13 can be applied to each face of this mesh.

[0019] Mesh data is composed of information such as that shown in the lower part of Figure 1. Vertex information 14, shown first from the left in the lower part of Figure 1, is information indicating the three-dimensional position (three-dimensional coordinates (X, Y, Z)) of each vertex 11 that constitutes the mesh. Connection information 15, shown second from the left in the lower part of Figure 1, is information indicating each connection (edge) 12 that constitutes the mesh. A texture image 16, shown third from the left in the lower part of Figure 1, is map information for the texture 13 that is applied to each face. A UV map 17, shown fourth from the left in the lower part of Figure 1, is information indicating the correspondence between the vertices 11 and the texture 13. The UV map 17 indicates the coordinates (UV coordinates) of each vertex 11 in the texture image 16.

[0020] As an example of such a mesh coding method, there is V-DMC (Video-based Dynamic Mesh Coding) as disclosed in Non-Patent Document 1.

[0021] In V-DMC, the mesh to be encoded (referred to in this specification as the original mesh) is represented as a base mesh that is less fine (i.e., coarser) than the original mesh, and displacement vectors of the division points obtained by subdividing the base mesh, and the base mesh and displacement vectors are then encoded.

[0022] For example, assume that there is an original mesh as shown in the top row of Figure 2. The original mesh is a mesh composed of vertices and connections that represent the three-dimensional structure of an object, and is the target of encoding. For example, the original mesh is generated from a captured image of an object in real space (by camera capture). In Figure 2, black dots represent vertices, and lines connecting the black dots represent connections (edges). As described above, a mesh essentially forms polygons using vertices and edges, but for convenience of explanation, it is described here as a group of vertices connected linearly (in series).

[0023] By simplifying the original mesh, a coarse (low-resolution) mesh like the one shown in the second row from the top of Figure 2 is formed. This is called the base mesh. One simplification method is to thin out some of the vertices (decimate). In other words, the base mesh is a mesh with lower resolution than the original mesh, generated by thinning out vertices from the original mesh (i.e., simplifying the original mesh).

[0024] By subdividing each polygon of this base mesh, vertices and edges are added, as shown in the third row from the top of Figure 2. The degree of subdivision is arbitrary. That is, the number of vertices and edges added is arbitrary. For example, this subdivision can add vertices equal to the number of vertices thinned out from the original mesh. That is, subdivision can be used to maintain the same number of vertices as the original mesh. In this specification, these added vertices are also referred to as division points. This subdivision can also be repeated recursively. For example, in a technique called midpoint, the process of adding vertices to the midpoints of edges (subdivision) is repeated recursively. In other words, recursive subdivision increases the number of vertices and improves the resolution of the mesh. In this way, it is possible to perform subdivision up to any desired level of resolution (i.e., control the resolution of the subdivided mesh). In other words, the subdivided mesh can be layered according to its level of resolution. In other words, this can be considered a layering of the subdivision process and the vertices (division points) and edges obtained by the subdivision process.

[0025] However, the connections of the base mesh are updated when the vertices of the original mesh are thinned out. Therefore, the division points obtained by subdivision are formed on these updated connections (edges). As a result, the shape of the subdivided base mesh differs from the shape of the original mesh. More specifically, as shown in the bottom part of Figure 2, the positions of the division points (on the dotted line) differ from those of the original mesh. In addition, the positions of the vertices of the base mesh may differ from those of the original mesh.

[0026] In other words, by moving the positions of the vertices of the subdivided base mesh (the vertices or division points of the base mesh) closer to the vertex positions of the original mesh, the difference in shape between the subdivided base mesh and the original mesh can be reduced. In this specification, such movement of the vertices of the subdivided base mesh (the vertices or division points of the base mesh) is also referred to as displacement. Furthermore, the amount and direction of this displacement, expressed as a vector, is also referred to as a displacement vector. Ideally, by displacing each vertex of the subdivided base mesh, the shape of the subdivided base mesh can be made to match the shape of the original mesh. In other words, the original mesh can be expressed as a base mesh and a displacement vector.

[0027] In V-DMC, such base meshes and displacement vectors are coded instead of the original mesh (geometry). By coding the base meshes and displacement vectors in this way, it is possible to code with a reduced number of polygons (i.e., the number of vertices and edges) compared to coding the original mesh, which generally reduces the amount of code for the same quality. In other words, it is possible to improve coding efficiency.

[0028] During decoding, as described above, a mesh is restored (generated) by subdividing a base mesh and applying a displacement vector to each vertex of the subdivided base mesh to displace it. In this specification, this mesh is also referred to as a decoded mesh. Attributes are then applied to the decoded mesh, and the decoded mesh with the applied attributes is rendered. While the shape of the polygon (face) may be any polygonal shape, the following description will be given assuming that the polygon is triangular. Therefore, the polygon (face) will also be referred to as a triangle in the following description.

[0029] <Encoding and Decoding of V-DMC Data> In the case of V-DMC, mesh data consists of a base mesh, displacement vectors, attributes, and atlas information. This data group is also referred to as V-DMC data. The base mesh consists of information indicating vertices and connections, and is coded using an existing mesh coding method such as Draco. Note that the base mesh can be coded not only intra-coded but also inter-coded. When inter-coding, the base mesh is coded as a motion vector between frames.

[0030] Displacement vectors are arithmetically coded or packed into a two-dimensional plane and encoded as video data using a coding method for 2D video. For example, when encoded as video data, displacement vectors are converted into displacement coefficients using a predetermined method. The displacement coefficients are arranged as pixel values ​​in a two-dimensional area (also called a displacement map). This arrangement (mapping) of displacement coefficients is also called packing. Video (also called displacement video) using the displacement map as frame images is encoded using a coding method for 2D video. In other words, displacement coefficients are scalar values ​​corresponding to the displacement vectors. A displacement map is map information (also called image data) that stores displacement coefficients as pixel values. Displacement video is video image data using the displacement map as frame images.

[0031] An attribute is non-geometry information applied to a mesh (geometry), which is 3D data. For example, an attribute may include a texture applied to a face of the mesh (geometry). The attribute (e.g., texture) is divided into multiple subregions, each of which is projected in a predetermined projection direction, and the projected images (patches) are arranged in a two-dimensional region (also called an attribute map). In other words, attribute patches are packed into the attribute map. A video (also called attribute video) using the attribute map as frame images is encoded using a 2D video encoding method. In other words, the attribute map is map information (also called image data) that stores the patches (projected textures) as pixel values. Attribute video is video data using the attribute map as frame images.

[0032] Atlas information is information used when reconstructing a mesh. For example, atlas information may include correspondence between the base mesh and a displacement map or attribute map (such as a UV map), quantized values ​​of displacement vectors, etc. This atlas information is encoded using a predetermined encoding method.

[0033] The coded data (bitstream) of each data is decoded by a decoding method corresponding to the coding method. In other words, by decoding the coded data (bitstream), various information such as base meshes, displacement vectors, attributes, and atlas information is restored (generated).

[0034] <Processing Load Related to Mesh Reconstruction and Subjective Quality of Mesh> Generally, the greater the amount of information in a decoded mesh, the greater the processing load related to the reconstruction of the decoded mesh. However, the smaller the amount of information in the decoded mesh, the lower the subjective quality of the decoded mesh. In this specification, the reconstruction of a decoded mesh (or sometimes referred to as mesh reconstruction) includes subdivision of a base mesh and application of a displacement vector to the subdivided base mesh. The reconstruction process may also include decoding of a bitstream of a displacement vector to obtain the displacement vector used in the reconstruction. The reconstruction process may also include decoding of a bitstream of a base mesh to obtain the base mesh used in the reconstruction. The "subdivision of a base mesh" may include any processing related to the subdivision of the base mesh (e.g., controlling the number of iterations, etc.). The "application of a displacement vector" may also include any processing related to the application of the displacement vector (e.g., unpacking of a displacement vector, etc.). Furthermore, "decoding a bitstream of a base mesh" and "decoding a bitstream of a displacement vector" may include any processing related to that decoding.

[0035] For example, mesh 31 in Figure 3A shows the state of the base mesh, and mesh 31 in Figure 3B shows the state after subdivision. That is, mesh 31 in Figure 3B has a larger number of faces (i.e., a larger amount of information) than mesh 31 in Figure 3A. As is clear from comparing Figure 3A with Figure 3B, the mesh 31 with a smaller number of faces (Figure 3A) has a coarser shape, while the mesh with a larger number of faces (Figure 3B) has a higher subjective quality. However, as the number of faces increases, the amount of information in mesh 31 increases, and the amount of processing required to reconstruct mesh 31 also increases. In other words, there is generally a trade-off between the subjective quality of a decoded mesh, which depends on the amount of information, and the amount of processing required to reconstruct that decoded mesh.

[0036] However, the impact of the amount of information on subjective quality is not necessarily constant and may vary depending on the part. For example, reducing the number of faces in a part with a complex shape results in a greater degradation (difference from the original mesh) and has a greater impact on subjective quality than reducing the number of faces in a part with a simple shape.

[0037] For example, in A of Fig. 3, mesh 31 represents the shape of a person, and region 32 of mesh 31 corresponds to the face of the person. Region 33 of mesh 31 corresponds to the hand of the person. A person's face and hands generally have more complex shapes than, for example, the torso or arms. In other words, if the number of faces in region 32 or region 33 decreases and the shape of the mesh becomes coarser, there is a risk that the subjective quality will be significantly reduced.

[0038] In other words, in order to suppress an increase in the processing load related to mesh reconstruction while suppressing a reduction in subjective quality, it is necessary to reduce the amount of information in parts of the decoded mesh that have less impact on subjective quality. In other words, it is necessary to prioritize the reconstruction of parts of the decoded mesh that have a greater impact on subjective quality. For example, by prioritizing the reconstruction of parts with relatively complex shapes and assigning a large number of faces to those parts, it is possible to suppress a reduction in subjective quality. In particular, when the performance of the decoding device is low and it is difficult to reconstruct the decoded mesh with the maximum amount of information, it is essential to reduce the amount of information, so it is important to determine which area of ​​the decoded mesh to prioritize reconstruction.

[0039] However, conventional methods lack a method for controlling which mesh region a decoding device prioritizes in reconstructing. In other words, the decoding device reconstructs the decoded mesh without considering subjective quality. As a result, for example, the decoding device may prioritize reconstructing a mesh portion that has a smaller impact on subjective quality, and may be unable to reconstruct a mesh portion that has a greater impact on subjective quality due to processing power limitations. For example, in mesh 31 in FIG. 3A, a case may be considered in which the reconstruction of a portion other than regions 32 and 33 is prioritized, resulting in a reduction in the amount of information in regions 32 and 33. In such a case, there is a risk that the subjective quality of that portion may be significantly reduced (i.e., the degradation may be noticeable). In other words, it is difficult to suppress the reduction in subjective quality while suppressing an increase in the processing load related to mesh reconstruction.

[0040] For example, there is a method of specifying the number of divisions for each submesh, but this method also does not control which area of ​​the mesh the decoding device will prioritize in reconstructing. Therefore, if it is difficult for the decoding device to reconstruct all the information of the mesh, there is a risk that the amount of information will be reduced without considering the subjective quality. In other words, it is difficult to suppress the reduction in subjective quality and the increase in the processing load related to the mesh reconstruction.

[0041] <3. Priority control of processing related to mesh reconstruction> <Method 1> Therefore, the decoding device is made able to control which region of the mesh is to be preferentially reconstructed. Figures 4 and 5 show examples of methods for controlling processing related to mesh reconstruction. As shown in the top row of the table in Figure 4, priority control information is transmitted, and a priority region of the mesh is preferentially reconstructed in accordance with the priority control information (Method 1). Note that in this specification, a non-priority region refers to a region other than a priority region, and a priority region refers to a region in which processing related to mesh reconstruction is given priority over a non-priority region.

[0042] For example, in the decoded mesh 101 (person) shown in A of FIG. 6, the region 102 corresponding to the face and the region 103 corresponding to the hand are relatively complex in shape. In other words, these regions are regions where subjective quality is significantly reduced by reducing the amount of information. Therefore, priority control information that controls the reconstruction of regions 102 and 103 with priority is transmitted, and the decoding device reconstructs regions 102 and 103 with priority in accordance with the priority control information. By doing so, for example, as shown in B of FIG. 6, it is possible to increase the number of faces in regions 102 and 103 in the decoded mesh 101 (i.e., increase the amount of information assigned), thereby suppressing a reduction in subjective quality. In other words, it is possible to suppress a reduction in subjective quality and also suppress an increase in the processing load related to mesh reconstruction.

[0043] Hereinafter, an information processing device (encoding device) that encodes 3D data including a base mesh and a displacement vector and generates a bitstream is also referred to as a first information processing device. For example, the first information processing device includes a priority control information generating unit that generates priority control information, which is information regarding the priority of processing related to mesh reconstruction, and an encoding unit that encodes the base mesh, the displacement vector, and the priority control information to generate a bitstream.

[0044] In addition, the first information processing device generates priority control information, which is information regarding the priority of processing related to mesh reconstruction, and encodes the base mesh, displacement vector, and the priority control information to generate a bit stream.

[0045] The base mesh is a mesh with lower resolution than the original mesh to be coded, which is composed of vertices and connections that represent the three-dimensional structure of the object, and is generated by thinning out vertices from the original mesh. The displacement vector is vector information that indicates the displacement of the vertices of the subdivided base mesh.

[0046] In this way, the first information processing device can use the priority control information to control which area of ​​the mesh the decoding device will preferentially reconstruct, which means that the first information processing device can suppress a decrease in subjective quality and suppress an increase in the processing load related to the reconstruction of the mesh.

[0047] In the following description, an information processing device (decoding device) that decodes a bit stream of 3D data including a base mesh and a displacement vector and reconstructs the decoded mesh is also referred to as a second information processing device. This second information processing device includes a priority control information decoding unit that decodes a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction, a subdivision unit that subdivides the base mesh as part of the reconstruction processing, and a displacement vector application unit that applies displacement vectors to the subdivided base mesh in accordance with the priority order based on the priority control information as part of the reconstruction processing.

[0048] In addition, the second information processing device decodes a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction, and as processing related to the reconstruction, subdivides the base mesh, and as processing related to the reconstruction, applies a displacement vector to the subdivided base mesh in accordance with the priority order based on the priority control information.

[0049] The base mesh is a mesh with lower resolution than the original mesh to be coded, which is composed of vertices and connections that represent the three-dimensional structure of the object, and is generated by thinning out vertices from the original mesh. The displacement vector is vector information that indicates the displacement of the vertices of the subdivided base mesh.

[0050] In this way, the second information processing device can prioritize the reconstruction of the target region specified by the priority control information, which means that the second information processing device can suppress a decrease in subjective quality and suppress an increase in the processing load related to the reconstruction of the mesh.

[0051] <Method 1-1> When Method 1 is applied, as shown in the second row from the top of the table in FIG. 4 , the priority control information may include priority information indicating the priority of processing related to the reconstruction of the target region (Method 1-1). For example, a priority control information generation unit of a first information processing device may generate priority control information including the priority information. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including the base mesh and the displacement vector. Furthermore, a priority control information decoding unit of a second information processing device may decode the bitstream to obtain priority control information including the priority information. Then, a displacement vector application unit may apply the displacement vector to the target region of the base mesh according to the priority information. This enables local reconstruction control according to priority.

[0052] <Method 1-1-1> When Method 1-1 is applied, the priority information may include a priority flag (Method 1-1-1), as shown in the third row from the top of the table in FIG. 4 . The priority flag is flag information indicating whether or not to prioritize the process related to the reconstruction of the target region. For example, a priority control information generation unit of a first information processing device may generate priority control information including the priority flag. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including a base mesh and a displacement vector. Furthermore, a priority control information decoding unit of a second information processing device may decode the bitstream to obtain the priority control information including the priority flag. Then, a displacement vector application unit may apply the displacement vector to a target region whose priority flag is true, in preference to a target region whose priority flag is false. This enables local reconstruction control based on a specification of whether or not to prioritize reconstruction.

[0053] Note that priority flags may be set for each partial region for the entire region of a mesh. For example, table information associating the ID of each partial region with the flag value of the priority flag corresponding to that partial region may be stored in the bitstream (e.g., [region 1, region 2, region 3, ...] = [1, 1, 0, ...]). Alternatively, only the priority flag corresponding to the priority region to be prioritized for reconstruction may be set. For example, a list of priority region IDs may be stored in the bitstream (e.g., [region 1, region 2]). Regions whose IDs are included in this list are treated as priority regions. In other words, this list can be considered to be information similar to priority flags.

[0054] <Method 1-1-2> When Method 1-1 is applied, the priority information may include information indicating the priority of processing related to the reconstruction of the target region, as shown in the fourth row from the top of the table in FIG. 4 (Method 1-1-2). For example, a priority control information generation unit of a first information processing device may generate priority control information including information indicating the priority of processing related to the reconstruction of the target region. An encoding unit may then encode the priority control information and store it in a bitstream of 3D data including a base mesh and a displacement vector. Furthermore, a priority control information decoding unit of a second information processing device may decode the bitstream to obtain priority control information including information indicating the priority of processing related to the reconstruction of the target region. A displacement vector application unit may then apply the displacement vector to the target region of the base mesh according to the priority. This enables local reconstruction control based on the specified priority.

[0055] The priority order may be specified, for example, as numbers 0 to n. For example, number 0 indicates an area that does not have priority for reconstruction (non-priority area). Numbers 1 to n indicate the priority order for the reconstruction process of that target area. This priority order may be set for each partial area for the entire mesh area. For example, table information that associates the ID of each partial area with the priority order of that partial area may be stored in the bitstream (e.g., [area 1, area 2, area 3, ...] = [2, 1, 0, ...] (0: non-priority area, 1 onwards: priority in order from 1 onwards)). Alternatively, only the priority order corresponding to the priority area may be set. For example, a list in which the IDs of priority areas are arranged in order of priority may be stored in the bitstream (e.g., [area 2, area 1]).

[0056] <Method 1-2> When Method 1 is applied, target area designation information may be included in the priority control information (Method 1-2), as shown in the fifth row from the top of the table in FIG. 4 . The target area designation information is information that designates a target area of ​​the priority control information. For example, a priority control information generation unit of a first information processing device may generate priority control information including the target area designation information. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including a base mesh and a displacement vector. Alternatively, a second information processing device may obtain priority control information including the target area designation information from the bitstream and identify the target area based on the target area designation information. This makes it easier to identify the target area.

[0057] <Method 1-2-1> When Method 1-2 is applied, the target area designation information may include mesh range designation information (Method 1-2-1), as shown in the sixth row from the top of the table in FIG. 4 . The mesh range designation information is information indicating the range of the target area in the mesh. For example, a priority control information generation unit of a first information processing device may generate priority control information including the mesh range designation information. An encoding unit may then encode the priority control information and store it in a bitstream of 3D data including the base mesh and displacement vectors. Alternatively, a second information processing device may obtain priority control information including the mesh range designation information from the bitstream and identify the range of the target area in the mesh based on the mesh range designation information. This makes it easier to identify the target area and more easily control local reconstruction.

[0058] <Method 1-2-1-1> When Method 1-2-1 is applied, the mesh range specification information may include a submesh ID (Method 1-2-1-1), as shown in the seventh row from the top of the table in Figure 4. The submesh ID is submesh identification information. In other words, in this case, the range of the target area in the mesh is specified in units of submeshes (i.e., submesh bitstreams). Figure 7 shows an example in which meshes in areas A, B, and C are each submeshed and encoded. Bitstream 111 is a bitstream obtained by encoding the submesh in area A. Bitstream 112 is a bitstream obtained by encoding the submesh in area B. Bitstream 113 is a bitstream obtained by encoding the submesh in area C. By dividing the bitstream for each submesh in this way, each submesh can be decoded independently from the others. When the bitstream is divided for each submesh in this way, the range of the target area in the mesh may be specified using the submesh ID as described above. For example, the priority control information generation unit of the first information processing device may generate priority control information including the submesh ID. The encoding unit may then encode the priority control information and store it in a bitstream of 3D data including the base mesh and displacement vector. Alternatively, the second information processing device may obtain the priority control information including the submesh ID from the bitstream and identify the bitstream corresponding to the submesh ID as the bitstream of the target region. This makes it easier to identify the target region and more easily control local reconstruction. Note that if the order of the decoded patches is uniquely determined, transmission of the submesh ID may be omitted.

[0059] <Method 1-2-1-2> When Method 1-2-1 is applied, the mesh range designation information may include a list of facegroupIDs (Method 1-2-1-2), as shown in the eighth row from the top of the table in FIG. 4 . FacegroupIDs are information described in Non-Patent Document 4, and are assigned to faces and indicate the face group (a set of faces) to which the faces belong. In other words, in this case, a portion of the bitstream (e.g., a portion of a submesh) can be designated as the target area range of the mesh. For example, by designating the bitstream 121 in FIG. 8 using facegroupIDs as described above, some meshes (e.g., area A, area B, area C, etc.) included in this bitstream 121 can be designated as the target area range of the mesh. Note that the bitstream 121 may correspond to any range of the mesh; for example, it may be a bitstream of a submesh or a bitstream of the entire area of ​​the mesh. For example, a priority control information generation unit of the first information processing device may generate priority control information including the list of facegroupIDs. The encoding unit may then encode the priority control information and store it in a bitstream of 3D data including the base mesh and displacement vectors. For example, the list of facegroup IDs may be stored in the bitstream as an attribute of the base mesh. Alternatively, the second information processing device may obtain priority control information including the list of facegroup IDs from the bitstream and identify the face group corresponding to the facegroup ID as the target region. This makes it easier to identify the target region and more easily control local reconstruction.

[0060] <Method 1-2-1-3> When Method 1-2-1 is applied, the mesh range specification information may include a list of mesh vertex or face indices (Method 1-2-1-3), as shown in the ninth row from the top of the table in Figure 4. In other words, in this case, a portion of a submesh (i.e., a submesh bitstream) can be specified as the mesh range. For example, a priority control information generation unit of a first information processing device may generate priority control information including a list of mesh vertex or face indices. An encoding unit may then encode the priority control information and store it in a bitstream of 3D data including a base mesh and displacement vectors. For example, this list may be stored in the bitstream as metadata. Alternatively, a second information processing device may obtain priority control information including the list from the bitstream and identify a region containing vertices or faces corresponding to the indices included in the list as a target region. This makes it easier to identify the target region and more easily control local reconstruction.

[0061] <Method 1-2-1-4> When Method 1-2-1 is applied, the mesh range specification information may include a list of the number of faces belonging to the target region of the mesh (Method 1-2-1-4), as shown in the tenth row from the top of the table in Figure 4. That is, in this case, a portion of a submesh (i.e., a bitstream of the submesh) can be specified as the mesh range. For example, a priority control information generation unit of a first information processing device may generate priority control information including a list of the number of faces belonging to the target region. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including a base mesh and displacement vectors. Alternatively, a second information processing device may obtain the priority control information including the list from the bitstream, group the faces in decoding order by the number of faces included in the list, and identify the group of faces as the target region. This makes it easier to specify the target region and more easily control local reconstruction.

[0062] <Method 1-2-2> When Method 1-2 is applied, the target area designation information may include displacement range designation information (Method 1-2-2), as shown in the 14th row from the top of the table in FIG. 4 . The displacement range designation information is information indicating the range of the target area in the displacement bitstream (bitstream of displacement vectors). In other words, the displacement range designation information designates a displacement vector corresponding to the target area. For example, a priority control information generation unit of a first information processing device may generate priority control information including the displacement range designation information. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including a base mesh and displacement vectors. Alternatively, a second information processing device may obtain priority control information including the displacement range designation information from the bitstream and identify the range of the target area in the displacement bitstream based on the displacement range designation information. This makes it easier to identify the displacement vector corresponding to the target area, thereby more easily controlling local reconstruction.

[0063] <Method 1-2-2-1> When Method 1-2-2 is applied, the displacement range designation information may include a displacement bitstream ID (Method 1-2-2-1), as shown in the 15th row from the top of the table in FIG. 4 . The displacement bitstream ID is identification information for the displacement bitstream. In other words, in this case, the range is designated in units of displacement bitstreams. For example, a priority control information generation unit of a first information processing device may generate priority control information including the displacement bitstream ID. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including a base mesh and a displacement vector. Alternatively, a second information processing device may obtain priority control information including the displacement bitstream ID from the bitstream and identify the displacement vector stored in the displacement bitstream corresponding to the displacement bitstream ID as the displacement vector corresponding to the target region. This makes it easier to identify the displacement vector corresponding to the target region, thereby more easily controlling local reconstruction.

[0064] <Method 1-2-2-2> When Method 1-2-2 is applied, the displacement range designation information may include a list of a starting pixel index or the number of pixels within the range, as shown in the 16th row from the top of the table in FIG. 4 (Method 1-2-2-2). The starting pixel index indicates the starting position (pixel) of the displacement vector corresponding to the target region in the two-dimensional plane in which the displacement vector is packed. The number of pixels within the range indicates the number of pixels within the target region. In other words, in this case, a portion of the displacement bitstream can be designated as the range. For example, a priority control information generation unit of the first information processing device may generate priority control information including the list of the starting pixel index or the number of pixels within the range. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including the base mesh and the displacement vector. Alternatively, a second information processing device may obtain the priority control information including the list from the bitstream and identify the displacement vector corresponding to the list as the displacement vector corresponding to the target region. For example, a displacement vector starting from the position (pixel) indicated by the starting pixel index may be identified as the displacement vector corresponding to the target region. Alternatively, the displacement vectors may be grouped by the number of pixels within the range, and the displacement vector corresponding to the target region may be identified for each group. This makes it easier to identify the displacement vector corresponding to the target region, and more easily control the local reconstruction. Note that if the number of pixels corresponding to each region is fixed, transmission of the list of the number of pixels within the range may be omitted, or a single fixed value may be transmitted as the number of pixels within the range.

[0065] <Method 1-2-2-3> When Method 1-2-2 is applied, the displacement range specification information may include a list of a starting block index or the number of blocks within the range, as shown in the bottom row of the table in FIG. 4 (Method 1-2-2-3). The starting block index indicates the starting position (block) of the displacement vector corresponding to the target area in the two-dimensional plane in which the displacement vector is packed. The number of blocks within the range indicates the number of blocks within the target area. In other words, in this case, a portion of the displacement bitstream can be specified as the range. For example, a priority control information generation unit of the first information processing device may generate priority control information including the list of the starting block index or the number of blocks within the range. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including the base mesh and the displacement vector. Alternatively, a second information processing device may obtain priority control information including the list from the bitstream and identify the displacement vector corresponding to the list as the displacement vector corresponding to the target area. For example, a displacement vector starting from the position (block) indicated by the start block index may be identified as the displacement vector corresponding to the target region. Alternatively, the displacement vectors may be grouped by the number of blocks within the range, and the displacement vector corresponding to the target region may be identified for each group. This makes it easier to identify the displacement vector corresponding to the target region, and more easily control the local reconstruction. Note that if the number of blocks corresponding to each region is fixed, transmission of the list of the number of blocks within the range may be omitted, or a single fixed value may be transmitted as the number of blocks within the range.

[0066] <Association Between Mesh Range Designation Information and Displacement Range Designation Information> When the target area designation information includes both mesh range designation information and displacement range designation information, the two pieces of information may be associated in the order of their lists, as shown in Fig. 9, for example. In the example of Fig. 9, a list of submesh IDs is applied as the mesh range designation information (Method 1-2-1-1), and a list of displacement bitstream IDs is applied as the displacement range designation information (Method 1-2-2-1). In this example, as shown in Fig. 9, the n-th elements in both lists may be associated with each other. Of course, any method of association may be used and is not limited to this example.

[0067] <Method 1-3> When Method 1 is applied, as shown in the top row of the table in FIG. 5, the displaced bitstream, which is a bitstream of displaced vectors, may have a configuration that allows the priority region to be decoded independently of the non-priority region (Method 1-3). For example, an encoding unit of a first information processing device may generate a displaced bitstream having such a configuration. Furthermore, a second information processing device may decode the priority bitstream included in the displaced bitstream independently of the non-priority bitstream. Note that the priority region indicates a region in which reconstruction-related processing is prioritized. The non-priority region indicates a region other than the priority region (i.e., a region in which reconstruction-related processing is not prioritized). The priority bitstream is a displaced bitstream corresponding to the priority region. The non-priority bitstream is a displaced bitstream corresponding to the non-priority region. This facilitates decoding of each region of the displaced bitstream, making it easier to control local reconstruction.

[0068] <Method 1-3-1> When Method 1-3 is applied, only the priority bitstream may be transmitted (Method 1-3-1), as shown in the second row from the top of the table in FIG. 5. That is, the transmitted displacement bitstream may include only the priority bitstream, as in the example shown in A of FIG. 10. For example, the encoding unit of the first information processing device may generate a displacement bitstream of such a configuration (i.e., a priority bitstream). That is, the encoding unit may generate a priority bitstream that encodes the displacement vectors of the priority region and store the priority bitstream as a displacement bitstream in the bitstream of the above-mentioned 3D data. Furthermore, the second information processing device may decode the displacement bitstream (priority bitstream). That is, the displacement vector decoding unit of the second information processing device may decode the priority bitstream, which is a bitstream of displacement vectors of a priority region that is prioritized over a non-priority region, as a reconstruction-related process, to generate a displacement vector. In this way, the displacement bitstream of the priority region can be decoded independently of the non-priority region. Then, the displacement vector application unit may apply the generated displacement vector to the subdivided base mesh in accordance with the priority order based on the priority control information.

[0069] <Method 1-3-2> When Method 1-3 is applied, as shown in the third row from the top of the table in FIG. 5, the displaced bitstream may have a prioritized bitstream and a non-priority bitstream that can be decoded independently of each other (Method 1-3-2). That is, the transmitted displaced bitstream may have a prioritized bitstream and a non-priority bitstream, as in the example shown in B of FIG. 10. The prioritized bitstream and the non-priority bitstream are each independent bitstreams and can be decoded independently of each other. For example, the encoding unit of the first information processing device may generate a displaced bitstream with such a configuration (i.e., a prioritized bitstream and a non-priority bitstream). That is, the encoding unit may generate a bitstream including a prioritized bitstream in which a displacement vector of a priority region is encoded and a non-priority bitstream in which a displacement vector of a non-priority region is encoded. Furthermore, the second information processing device may decode the displaced bitstream (priority bitstream and non-priority bitstream). In other words, the disparity vector decoding unit of the second information processing device may decode a disparity bitstream including a priority bitstream and a non-priority bitstream as a reconstruction process to generate a disparity vector. In this way, the disparity bitstream of the priority region can be decoded independently of the non-priority region. Then, the disparity vector application unit may apply the generated disparity vector to the subdivided base mesh in accordance with the priority order based on the priority control information.

[0070] <Method 1-3-3> When Method 1-3 is applied, as shown in the fourth row from the top of the table in FIG. 5 , the displaced bitstream may include a full-region bitstream and a priority bitstream that can be decoded independently of the full-region bitstream (Method 1-3-3). The full-region bitstream is a displaced bitstream corresponding to the full region (priority region and non-priority region). That is, the transmitted displaced bitstream may include a priority bitstream and a full-region bitstream, as in the example shown in A of FIG. 11 . The priority bitstream and the full-region bitstream are independent bitstreams that can be decoded independently of each other. For example, the encoding unit of the first information processing device may generate a displaced bitstream with such a configuration (i.e., the priority bitstream and the full-region bitstream). That is, the encoding unit may generate a bitstream that includes a priority bitstream that encodes the displacement vector of the priority region and a full-region bitstream that encodes the displacement vector of the full region. Furthermore, the second information processing device may decode the displaced bitstream (priority bitstream and full-region bitstream). In other words, the disparity vector decoding unit of the second information processing device may decode the disparity bitstream including the priority bitstream and the full-region bitstream as a reconstruction process to generate a disparity vector. In this way, the disparity bitstream of the priority region can be decoded independently of the disparity bitstream of the non-priority region. Then, the disparity vector application unit may apply the generated disparity vector to the subdivided base mesh in accordance with the priority order based on the priority control information.

[0071] Note that the full-region bitstream also includes information about the priority region. In this case, the information about the priority bitstream and the information about the priority region included in the full-region bitstream may completely overlap (may be identical to each other). In this case, when only the priority region is reconstructed, the priority bitstream is decoded, and when the full region is reconstructed, the full-region bitstream is decoded. Also, the information about the priority region included in the full-region bitstream may be a difference from the information about the priority bitstream. In this case, when only the priority region is reconstructed, the priority bitstream is decoded, and when the full region is reconstructed, the priority bitstream and the full-region bitstream are decoded. In other words, in this case, the displacement vectors of the priority region obtained when only the priority region is reconstructed are different from those obtained when the full region is reconstructed.

[0072] <Method 1-3-4> When Method 1-3 is applied, as shown in the fifth row from the top of the table in FIG. 5, the displaced bitstream may include a full-region bitstream configured to enable the priority region and the non-priority region to be decoded independently (Method 1-3-4). That is, the transmitted displaced bitstream may include only a full-region bitstream, as in the example shown in B of FIG. 11. In this case, the full-region bitstream has the displacement vectors of the priority region and the displacement vectors of the non-priority region packed independently of each other. Therefore, the portion of the full-region bitstream corresponding to the priority region can be decoded independently of the portion corresponding to the non-priority region. For example, the encoding unit of the first information processing device may generate a displaced bitstream configured in this way (i.e., a full-region bitstream). That is, the encoding unit of the first information processing device may generate a bitstream including a full-region bitstream that encodes the displacement vectors of the full region, in which the displacement vectors of the priority region and the displacement vectors of the non-priority region are packed independently of each other. Furthermore, the second information processing device may decode the portion of the displaced bitstream (full-region bitstream) corresponding to the priority region. In other words, the displacement vector decoding unit of the second information processing device may generate a displacement vector by decoding a bit stream of the entire region, in which the displacement vectors of the priority region and the displacement vectors of the non-priority region are packed independently of each other, as a reconstruction process. In this way, the displacement bit stream of the priority region can be decoded independently of the non-priority region. Then, the displacement vector application unit may apply the generated displacement vector to the subdivided base mesh in accordance with the priority order based on the priority control information.

[0073] <Method 1-4> When Method 1 is applied, as shown in the sixth row from the top of the table in FIG. 5, a predetermined process among the processes related to reconstruction for the priority area may be prioritized in accordance with the priority control information (Method 1-4). In other words, the priority control information may be information regarding the priority of a predetermined process among the processes related to reconstruction for the target area. This predetermined process may be any process as long as it is included in the processes related to reconstruction. In other words, all processes included in the processes related to reconstruction may be prioritized, or any part of the processes related to reconstruction may be prioritized.

[0074] <Method 1-4-1> When Method 1-4 is applied, as shown in the seventh row from the top of the table in FIG. 5 , priority may be given to subdividing the base mesh (Method 1-4-1). That is, the priority control information may include information regarding the priority of subdividing the base mesh. For example, a priority control information generation unit of a first information processing device may generate priority control information, which is information regarding the priority of subdividing the base mesh. An encoding unit may then encode the priority control information and store it in a bitstream of 3D data including the base mesh and displacement vectors. Furthermore, a priority control information decoding unit of a second information processing device may decode the bitstream to obtain the priority control information. Then, a subdivision unit may subdivide the priority region of the base mesh specified by the priority control information with priority over non-priority regions of the base mesh. In this way, control regarding the priority of subdividing the base mesh can be realized.

[0075] For example, the decoding device (second information processing device) may subdivide a priority region specified by the priority control information of the base mesh and apply a displacement vector to the subdivided priority region. In this case, the decoding device (second information processing device) may decode the displacement bitstream for the entire region and apply the displacement vector corresponding to the priority region among the obtained displacement vectors for the entire region to the priority region (i.e., the subdivided portion) of the base mesh. The decoding device (second information processing device) may also decode the displacement bitstream for the priority region and apply the obtained displacement vector corresponding to the priority region to the priority region (i.e., the subdivided portion) of the base mesh. After processing the priority region in this manner, if a limit, such as the number of faces, is not exceeded, the decoding device (second information processing device) may additionally perform processing (within the limits) such as subdividing a non-priority region, decoding the displacement bitstream for the non-priority region, and applying the displacement vector corresponding to the non-priority region (i.e., processing related to the reconstruction of the non-priority region). The decoding device (second information processing device) may also perform zippering on region boundaries as described in Non-Patent Document 5.

[0076] <Method 1-4-2> When Method 1-4 is applied, as shown in the eighth row from the top of the table in FIG. 5, the decoding of the displacement vector may be prioritized (Method 1-4-2). That is, the priority control information may include information regarding the priority of decoding of the displacement vector. For example, a priority control information generation unit of the first information processing device may generate priority control information, which is information regarding the priority of decoding of the displacement vector corresponding to the target region. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including the base mesh and the displacement vector. Alternatively, a priority control information decoding unit of the second information processing device may decode the bitstream and obtain the priority control information. Then, as a reconstruction process, a displacement vector decoding unit may decode a displacement bitstream, which is a bitstream of the displacement vectors, and generate displacement vectors for the priority region specified by the priority control information with priority over displacement vectors for the non-priority region. Then, a displacement vector application unit may apply the generated displacement vectors to the subdivided base mesh. In this way, control regarding the priority of decoding of the displacement vectors can be realized.

[0077] For example, the decoding device (second information processing device) may subdivide the base mesh of the entire region, decode the displacement bitstream of the priority region specified by the priority control information, and apply the obtained displacement vector corresponding to the priority region to the subdivided base mesh. Then, if the limit of, for example, the number of faces is not exceeded after processing the priority region, the decoding device (second information processing device) may additionally perform processing (within the range not exceeding the limit) such as subdividing the non-priority region, decoding the displacement bitstream of the non-priority region, and applying the displacement vector corresponding to the non-priority region (i.e., processing related to the reconstruction of the non-priority region).

[0078] <Method 1-4-3> When Method 1-4 is applied, priority may be given to applying a displacement vector to a subdivided base mesh (Method 1-4-3), as shown in the bottom row of the table in FIG. 5 . That is, a predetermined process among the reconstruction processes may include applying a displacement vector to a subdivided base mesh. For example, a priority control information generation unit of a first information processing device may generate priority control information including information regarding the priority of applying a displacement vector. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including the base mesh and the displacement vector. Furthermore, a priority control information decoding unit of a second information processing device may decode the bitstream to obtain the priority control information. Furthermore, a displacement vector application unit may apply a displacement vector of a priority region specified by the priority control information to a subdivided base mesh, giving priority to the displacement vector of a non-priority region. In this way, control regarding the priority of applying a displacement vector to a subdivided base mesh can be realized.

[0079] For example, the decoding device (second information processing device) may subdivide the base mesh of the entire region, decode the displacement bitstream of the entire region, and apply the displacement vector corresponding to the priority region specified by the priority control information among the obtained displacement vectors to the priority region of the subdivided base mesh. Then, if the limit of, for example, the number of faces is not exceeded after processing of the priority region, the decoding device (second information processing device) may additionally perform processing (within the range not exceeding the limit) such as subdividing the non-priority region, decoding the displacement bitstream of the non-priority region, and applying the displacement vector corresponding to the non-priority region (i.e., processing related to the reconstruction of the non-priority region).

[0080] 4. Local Control of Reconstruction-Related Processing Using Texture Information Local Control of Reconstruction-Related Processing Conventional V-DMC techniques divide an original mesh into local regions and encode them as submeshes, thereby enabling reconstruction control for each submesh. Here, "control of reconstruction" refers to control of processing included in the reconstruction-related processing. For example, this reconstruction control may include controlling the number of divisions, controlling the decoding and application of displacement vectors, and controlling the encoding method (intra- or inter-). The control performance (e.g., the size of the data unit to be controlled) of such submesh-specific control depends on the number of submeshes. In other words, when controlling the reconstruction-related processing of a mesh, the fewer the number of submeshes corresponding to that mesh, the larger the control unit (the data unit to be controlled) becomes, which may result in coarser control (i.e., reduced reconstruction control performance). In other words, dividing a mesh into more submeshes allows the control unit (the data unit to be controlled) to be smaller. This enables more precise control, improving the reconstruction control performance.

[0081] However, increasing the number of sub-meshes in this way to suppress the degradation of the performance of the reconstruction control increases the amount of code, which may reduce the coding efficiency. In other words, it is difficult to suppress the degradation of the coding efficiency and the degradation of the performance of the reconstruction control at the same time.

[0082] <Method 2> Therefore, as shown in the top row of the table in FIG. 12 , reconstruction control information including target region designation information and control information that utilizes texture information is transmitted (Method 2). In the case of V-DMC, the texture of a mesh is packed into a two-dimensional plane as an attribute and encoded. That is, a texture corresponding to a mesh in three-dimensional space, such as that shown in FIG. 13A, is transmitted as two-dimensional planar texture information, such as that shown in FIG. 13B. That is, a region in a mesh in three-dimensional space can be specified by such two-dimensional planar texture information. For example, mesh 161 in FIG. 13A is assumed to be a mesh in the priority region. Furthermore, the texture information of this mesh 161 corresponds to patches 162 and 163 in FIG. 13B. In V-DMC, information associating these is transmitted, for example, as atlas information, a base mesh, or the like. Therefore, mesh 161, which is the priority region, can be identified using patches 162 and 163 (i.e., texture information). That is, these patches can be simply specified. That is, the IDs of these patches may be transmitted as target area designation information.

[0083] Hereinafter, an information processing device (encoding device) that encodes 3D data including a base mesh and a displacement vector and generates a bitstream is also referred to as a third information processing device. For example, this third information processing device includes a target region designation information generation unit that generates target region designation information that designates a target region for controlling processing related to mesh reconstruction, a control information generation unit that generates control information that indicates control of processing related to reconstruction for the target region, and an encoding unit that encodes the base mesh, the displacement vector, and reconstruction control information including the target region designation information and the control information, and generates a bitstream.

[0084] In addition, in a third information processing device, target area designation information that designates a target area for controlling processing related to mesh reconstruction is generated, control information that indicates control of processing related to reconstruction for that target area is generated, and reconstruction control information that includes the base mesh, displacement vector, and the target area designation information and the control information is encoded to generate a bitstream.

[0085] The base mesh is a mesh with lower resolution than the original mesh to be coded, which is composed of vertices and connections that represent the three-dimensional structure of the object, and is generated by thinning out vertices from the original mesh. The displacement vector is vector information that indicates the displacement of the vertices of the subdivided base mesh. The target area designation information designates the target area using texture information related to the texture of the mesh.

[0086] By doing so, even if the target region is made smaller to suppress a decrease in the performance of the reconstruction control, it is possible to suppress an increase in the amount of code compared to the case of sub-mesh units. In other words, the third information processing device can suppress a decrease in the coding efficiency and also suppress a decrease in the performance of the reconstruction control.

[0087] In the following, an information processing device (decoding device) that decodes a bit stream of 3D data including a base mesh and a displacement vector and reconstructs the decoded mesh is also referred to as a fourth information processing device. This fourth information processing device includes a reconstruction control information decoding unit that decodes a bit stream of reconstruction control information including target area designation information that designates a target area for control of processing related to mesh reconstruction and control information that indicates control of processing related to reconstruction for the target area, a base mesh decoding unit that performs a base mesh decoding process to decode the base mesh bit stream as the processing related to the reconstruction, a subdivision unit that performs a subdivision process to subdivide the decoded base mesh as the processing related to the reconstruction, a displacement vector decoding unit that performs a displacement vector decoding process to decode the displacement vector bit stream as the processing related to the reconstruction, and a displacement vector application unit that performs a displacement vector application process to apply the decoded displacement vector to the subdivided base mesh as the processing related to the reconstruction.

[0088] In addition, in a fourth information processing device, the following processes are performed: decoding a bit stream of reconstruction control information including target area designation information that specifies a target area for control of processing related to mesh reconstruction and control information that indicates control of processing related to reconstruction for that target area; as processing related to the reconstruction, performing a base mesh decoding process that decodes the bit stream of the base mesh; as processing related to the reconstruction, performing a subdivision process that subdivides the base mesh obtained by decoding; as processing related to the reconstruction, performing a displacement vector decoding process that decodes the bit stream of the displacement vector; and as processing related to the reconstruction, performing a displacement vector application process that applies the displacement vector obtained by decoding to the subdivided base mesh.

[0089] The base mesh is a mesh with lower resolution than the original mesh to be coded, which is composed of vertices and connections that represent the three-dimensional structure of the object, and is generated by thinning out vertices from the original mesh. The displacement vector is vector information indicating the displacement of the vertices of the subdivided base mesh. The target area designation information specifies the target area using texture information related to the texture of the mesh. At least one of the base mesh decoding process, subdivision process, displacement vector decoding process, and displacement vector application process is then performed in accordance with the reconstruction control information.

[0090] By doing so, even if the target region is made smaller to suppress a decrease in the performance of the reconstruction control, it is possible to suppress an increase in the amount of code compared to the case of sub-mesh units. In other words, the fourth information processing device can suppress a decrease in the coding efficiency and also suppress a decrease in the performance of the reconstruction control.

[0091] <Method 2-1> When Method 2 is applied, the target area designation information may include a UV connected area ID, as shown in the second row from the top of the table in Figure 12 (Method 2-1). The UV connected area ID is identification information for the UV connected area. A UV connected area is a collection (block) of textures for each face on a two-dimensional plane where textures are packed. In a UV connected area, the texture for each face is arranged so that it shares an edge with the texture of the adjacent face (connected to the adjacent face). In other words, a UV connected area is made up of a group of triangles that share an edge. For example, patches 162 and 163 in Figure 13B are UV connected areas.

[0092] One method for arranging patches on a two-dimensional plane is to explicitly transmit coordinates of the two-dimensional plane (also referred to as UV coordinates), such as a UV atlas. In this method, the base mesh transmitted to the decoding device includes information such as that shown in A of FIG. 14 . For example, this base mesh 171 includes UV connectivity information (UV Triangle list) shown in a rectangular box 172. This UV connectivity information (UV Triangle list) defines faces (TTn) on the two-dimensional plane. Therefore, this UV connectivity information (UV Triangle list) can be used to represent a UV-connected region. In other words, by using the UV connectivity information (UV Triangle list), a target region in a mesh in three-dimensional space can be identified from the UV-connected region.

[0093] For example, a target region designation information generation unit of a third information processing device may generate target region designation information including a UV-connected region ID. An encoding unit may then encode the target region designation information and store it in a bitstream of 3D data including a base mesh and a displacement vector. Alternatively, a fourth information processing device may acquire the UV-connected region ID included in the target region designation information from the bitstream and identify a target region in a mesh in a three-dimensional space by associating the UV-connected region ID with a face (TTn) in the UV connectivity information based on the processing order. By doing so, even if the target region is made smaller to prevent a reduction in the performance of reconstruction control, the increase in the amount of code can be reduced compared to when the target region is processed in sub-mesh units, since only the number of UV-connected region IDs included in the target region designation information increases. In other words, a reduction in encoding efficiency can be prevented, and a reduction in the performance of reconstruction control can be prevented.

[0094] Note that some of the information described as being transmitted as the base mesh 171 in FIG. 14A, such as UV connectivity information, may be transmitted as atlas information.

[0095] <Method 2-2> When Method 2 is applied, the target area designation information may include an orthoAtlas patch ID, as shown in the third row from the top of the table in FIG. 12 (Method 2-2). The orthoAtlas patch ID is identification information for a patch on a two-dimensional plane where texture is packed by the orthoAtlas. For example, a method for arranging patches on a two-dimensional plane, such as orthoAtlas, does not explicitly transmit coordinates (also referred to as UV coordinates) on the two-dimensional plane. In this method, the base mesh transmitted to the decoding device includes information such as that shown in B of FIG. 14. For example, this base mesh 173 includes a UV list shown in a rectangular box 174. This UV list defines the ID (CCID) of the patch (Connected Component) to which each vertex of the mesh belongs. Therefore, by using this UV list, the target area in a mesh in three-dimensional space can be identified from the orthoAtlas patch.

[0096] For example, a target region designation information generation unit of a third information processing device may generate target region designation information including an orthoAtlas patch ID (CCID). An encoding unit may then encode the target region designation information and store it in a bitstream of 3D data including a base mesh and a displacement vector. Alternatively, a fourth information processing device may obtain the orthoAtlas patch ID (CCID) included in the target region designation information from the bitstream and identify the target region in a mesh in 3D space by associating the orthoAtlas patch ID (CCID) with vertices based on the UV list. By doing so, even if the target region is made smaller to prevent a decrease in reconstruction control performance, the increase in the amount of code can be suppressed compared to when the target region is processed in sub-mesh units, since only the number of orthoAtlas patch IDs (CCIDs) included in the target region designation information increases. In other words, a decrease in encoding efficiency can be suppressed while a decrease in reconstruction control performance can be suppressed.

[0097] Note that some of the information described as being transmitted as the base mesh 173 in FIG. 14B, such as a UV list, may be transmitted as atlas information.

[0098] <Method 2-3> When Method 2 is applied, the control information may include information for controlling a predetermined process among the processes related to the reconstruction, as shown in the fourth row from the top of the table in Fig. 12 (Method 2-3). This predetermined process may be any process included in the processes related to the reconstruction. In other words, all processes included in the processes related to the reconstruction may be controlled, or any part of the processes related to the reconstruction may be controlled.

[0099] <Method 2-3-1> When Method 2-3 is applied, as shown in the fifth row from the top of the table in FIG. 12 , a predetermined process among the reconstruction-related processes to be controlled may include subdivision of the base mesh (Method 2-3-1). For example, a control information generation unit of a third information processing device may generate control information for controlling the subdivision of the base mesh (e.g., the number of divisions, etc.). Then, an encoding unit may encode reconstruction control information including the control information and store the encoded information in a bitstream of 3D data including the base mesh and displacement vectors. Furthermore, a reconstruction control information decoding unit of a fourth information processing device may decode the bitstream of the reconstruction control information and acquire the encoded control information. Then, a subdivision unit may control the subdivision of the base mesh (e.g., the number of divisions, etc.) based on the control information. That is, the subdivision unit may perform subdivision processing on the target region of the base mesh in accordance with the control information. For example, the subdivision unit may subdivide the target region of the base mesh according to the number of divisions (number of iterations) specified by the control information. In this manner, control of the subdivision of the base mesh can be achieved.

[0100] <Method 2-3-2> When Method 2-3 is applied, as shown in the sixth row from the top of the table in FIG. 12 , the predetermined process among the reconstruction-related processes to be controlled may include decoding of the displacement vector (Method 2-3-2). For example, a control information generation unit of a third information processing device may generate control information for controlling the decoding of the displacement vector (e.g., priority order, etc.). Then, an encoding unit may encode reconstruction control information including the control information and store it in a bitstream of 3D data including the base mesh and the displacement vector. Alternatively, a reconstruction control information decoding unit of a fourth information processing device may decode the bitstream of the reconstruction control information and acquire the control information. Then, a displacement vector decoding unit may control the decoding of the displacement vector (e.g., priority order, etc.) based on the control information. That is, the displacement vector decoding unit may perform a displacement vector decoding process for the displacement vector of the target region in accordance with the control information. For example, the displacement vector decoding unit may decode the displacement bitstream of the target region of the control information in accordance with the priority order specified by the control information and acquire the displacement vector corresponding to the target region. In this way, control of the decoding of the displacement vector can be achieved.

[0101] <Method 2-3-3> When Method 2-3 is applied, as shown in the seventh row from the top of the table in FIG. 12 , a predetermined process among the reconstruction-related processes to be controlled may include applying a displacement vector to a base mesh (Method 2-3-3). For example, a control information generation unit of a third information processing device may generate control information for controlling the application of a displacement vector to a base mesh (e.g., priority order, etc.). Then, an encoding unit may encode reconstruction control information including the control information and store the encoded information in a bitstream of 3D data including the base mesh and the displacement vector. Alternatively, a reconstruction control information decoding unit of a fourth information processing device may decode the bitstream of the reconstruction control information and acquire the control information. Then, a displacement vector application unit may control the application of a displacement vector to a base mesh (e.g., priority order, etc.) based on the control information. That is, the displacement vector application unit may perform a displacement vector application process for the displacement vector of the target region in accordance with the control information. For example, the displacement vector application unit may apply a displacement vector corresponding to the target region of the control information to the target region of the subdivided base mesh in accordance with the priority order specified in the control information. In this way, control of the application of the displacement vector can be achieved.

[0102] <Method 2-3-4> When Method 2-3 is applied, as shown in the bottom row of the table in FIG. 12 , the predetermined process among the reconstruction-related processes to be controlled may include decoding of the base mesh (Method 2-3-4). For example, a control information generation unit of the third information processing device may generate control information for controlling the decoding of the base mesh (e.g., the encoding method (intra-, inter-, etc.) and encoding parameters). Then, an encoding unit may encode reconstruction control information including the control information and store the encoded information in a bitstream of 3D data including the base mesh and displacement vectors. Alternatively, a reconstruction control information decoding unit of the fourth information processing device may decode the bitstream of the reconstruction control information to obtain the control information. Then, a base mesh decoding unit may control the decoding of the base mesh (e.g., the encoding method (intra-, inter-, etc.) and encoding parameters) based on the control information. That is, the base mesh decoding unit may perform base mesh decoding processing for the target region of the base mesh in accordance with the control information. For example, the base mesh decoding unit may use the encoding method (intra, inter, etc.) specified by the control information to decode the bit stream of the base mesh corresponding to the target area of ​​the control information. In this way, the control of the decoding of the base mesh can be realized.

[0103] <5. Priority Control of Processing Related to Reconstruction Using Texture Information> <Scope of Application of Explanation> In this specification, an explanation given for a higher-level method also applies to lower-level methods belonging to that method, unless a contradiction arises. For example, when it is explained that "method 1 may be applied," this means that methods 1-1, 1-2, 1-3, and 1-4 may be applied. Of course, this also means that lower-level methods (e.g., method 1-1-1) may be applied. The same applies to method 2.

[0104] <Combination> Each of the above-described methods may be applied in combination with any other method as long as no contradiction occurs. Three or more methods may be applied in combination. Furthermore, techniques that can be combined may include not only those shown as "methods" in the tables of Figures 4, 5, and 12, but also all elements described in this specification. Furthermore, each of the above-described methods may be applied in combination with methods other than those described above.

[0105] <Method 1-2-1-5> For example, the above-described methods 1 and 2 may be combined and applied. That is, in controlling the priority of reconstruction processing, a target region may be specified using texture information related to the mesh texture. That is, priority control information including target region designation information and control information using the texture information may be transmitted. For example, when method 1-2-1 is applied, as shown in the eleventh row from the top of the table in FIG. 4, mesh range designation information may specify the mesh range using texture information (Method 1-2-1-5). In this case, a portion of a submesh (i.e., a submesh bitstream) may be designated as the mesh range. For example, a priority control information generation unit of a first information processing device may generate priority control information including mesh range designation information that specifies the target region using texture information related to the mesh texture. Then, an encoding unit may encode the priority control information and store it in a bitstream of 3D data including a base mesh and a displacement vector. Furthermore, a second information processing device may decode a bitstream of priority control information, which is information related to the priority of processing related to mesh reconstruction, acquire the priority control information, and identify the target region using texture information based on the priority control information. This makes it easier to identify the target region and more easily control the local reconstruction process, and also makes it possible to suppress a decrease in coding efficiency and a decrease in the performance of reconstruction control.

[0106] <Method 1-2-1-5-1> When Method 1-2-1-5 is applied, the mesh range designation information may include a UV-connected region ID, as shown in the twelfth row from the top of the table in FIG. 4 (Method 1-2-1-5-1). For example, a priority control information generation unit of a first information processing device may generate priority control information including a UV-connected region ID (mesh range designation information). An encoding unit may then encode the priority control information and store it in a bitstream of 3D data including a base mesh and displacement vectors. Alternatively, a second information processing device may decode a bitstream of priority control information, which is information regarding the priority of processing related to mesh reconstruction, obtain the UV-connected region ID included in the priority control information, and identify a target region in a mesh in three-dimensional space by associating the UV-connected region ID with a face (TTn) in the UV connectivity information based on the processing order. This makes it easier to identify the target region and more easily control processing related to local reconstruction. Furthermore, it is possible to suppress a decrease in encoding efficiency and a decrease in the performance of reconstruction control.

[0107] <Method 1-2-1-5-2> When Method 1-2-1-5 is applied, the mesh range specification information may include an orthoAtlas patch ID (Method 1-2-1-5-2), as shown in the thirteenth row from the top of the table in FIG. 4 . For example, a priority control information generation unit of a first information processing device may generate priority control information including an orthoAtlas patch ID (mesh range specification information). An encoding unit may then encode the priority control information and store it in a bitstream of 3D data including a base mesh and displacement vectors. Alternatively, a second information processing device may decode a bitstream of priority control information, which is information regarding the priority of processing related to mesh reconstruction, obtain the orthoAtlas patch ID included in the priority control information, and identify a target area in a mesh in 3D space by associating the orthoAtlas patch ID (CCID) with a vertex based on the UV list. This makes it easier to identify the target area and more easily control processing related to local reconstruction. Furthermore, it is possible to suppress a decrease in encoding efficiency and a decrease in reconstruction control performance.

[0108] <6. First embodiment> <Encoding device> The present technology can be applied to any device. For example, the present technology can be applied to an encoding device that encodes a mesh and generates a bitstream. FIG. 15 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology is applied. The encoding device 300 (first information processing device) shown in FIG. 15 is a device that encodes a mesh and generates a bitstream thereof. Therefore, the encoding device 300 can also be said to be a bitstream generation device that generates a bitstream.

[0109] Fig. 15 shows the main processing units, data flows, etc., but is not limited to all that is shown in Fig. 15. In other words, in encoding device 300, there may be processing units that are not shown as blocks in Fig. 15, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 15.

[0110] The encoding device 300 encodes a mesh using a method essentially similar to the V-DMC method described in the aforementioned non-patent document, except that the present technology is applied. For example, the encoding device 300 acquires an original mesh to be encoded and an attribute map including a texture corresponding to the original mesh. Note that the original mesh includes not only information about the mesh's geometry but also information indicating the correspondence with the attribute map (e.g., a UV list). The encoding device 300 encodes the original mesh and attribute map using the V-DMC method, generates a V-DMC bitstream, and outputs it.

[0111] 15 , the encoding device 300 (first information processing device) has a preprocessing unit 311 and a V-DMC encoding unit 312. The preprocessing unit 311 performs preprocessing before encoding. As shown in FIG. 15 , the preprocessing unit 311 has a base mesh generation unit 321, an atlas information generation unit 322, a priority control information generation unit 323, and a displacement vector generation unit 324.

[0112] The base mesh generation unit 321 performs processing related to the generation of a base mesh. For example, the base mesh generation unit 321 may obtain an original mesh to be input to the encoding device 300. The base mesh generation unit 321 may also perform decimation processing (thinning out vertices) on the original mesh to generate a base mesh. The base mesh generation unit 321 may supply the generated base mesh together with the original mesh to the atlas information generation unit 322.

[0113] The atlas information generation unit 322 performs processing related to the generation of atlas information corresponding to a base mesh. For example, the atlas information generation unit 322 may acquire a base mesh or an original mesh supplied from the base mesh generation unit 321. The atlas information generation unit 322 may generate atlas information by, for example, UV unwrapping the base mesh. The atlas information generation unit 322 may exchange information with the priority control information generation unit 323. For example, the atlas information generation unit 322 may acquire priority control information supplied from the priority control information generation unit 323. The atlas information generation unit 322 may include the priority control information in the atlas information. The atlas information generation unit 322 may supply the generated atlas information (including the priority control information) to the displacement vector generation unit 324 together with the base mesh, etc.

[0114] The priority control information generation unit 323 executes processing related to the generation of priority control information. For example, the priority control information generation unit 323 may generate priority control information by applying the above-described method 1. Furthermore, when generating the priority control information, the priority control information generation unit 323 may acquire information such as a base mesh or an original mesh supplied from the atlas information generation unit 322 and use the base mesh or the original mesh. The priority control information generation unit 323 may supply the generated priority control information to the atlas information generation unit 322.

[0115] The displacement vector generation unit 324 performs processing related to the generation of displacement vectors. For example, the displacement vector generation unit 324 may acquire a base mesh, atlas information, etc. supplied from the atlas information generation unit 322. Alternatively, the displacement vector generation unit 324 may acquire an original mesh input to the encoding device 300. Alternatively, the displacement vector generation unit 324 may use this information to generate displacement vectors that displace the vertices of the subdivided base mesh. The displacement vector generation unit 324 may supply the generated displacement vectors to the V-DMC encoding unit 312 together with the base mesh, atlas information (including priority control information), etc.

[0116] The V-DMC encoder 312 performs processing related to encoding of V-DMC data. For example, the V-DMC encoder 312 may acquire an original mesh input to the encoding device 300. The V-DMC encoder 312 may also acquire a base mesh, a displacement vector, atlas information (including priority control information), and the like, supplied from the displacement vector generator 324. The V-DMC encoder 312 may also acquire an attribute map input to the encoding device 300. The V-DMC encoder 312 may use this information to encode the atlas information (including priority control information), the base mesh, the displacement vector, and the attribute map, respectively, and generate the respective encoded data. Therefore, the V-DMC encoder 312 can also be referred to as an encoder. The V-DMC encoder 312 may also multiplex these encoded data as substreams to generate a single bitstream. This bitstream is also referred to as a V-DMC bitstream. Therefore, the V-DMC encoder 312 can also be called a bitstream generator (or a V-DMC bitstream generator). The V-DMC encoder 312 may output the generated V-DMC bitstream to the outside of the encoding device 300.

[0117] <V-DMC Encoder> Fig. 16 is a block diagram showing an example of the main configuration of the V-DMC encoder 312. Note that Fig. 16 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 16. In other words, the V-DMC encoder 312 may include processing units that are not shown as blocks in Fig. 16, and may include processing and data flows that are not shown as arrows or the like in Fig. 16.

[0118] As shown in FIG. 16, the V-DMC encoding unit 312 has an atlas information encoding unit 351, a base mesh encoding unit 352, a displacement vector correction unit 353, a displacement vector encoding unit 354, a mesh reconstruction unit 355, an attribute map conversion unit 356, an attribute encoding unit 357, and a multiplexing unit 358.

[0119] The atlas information encoder 351 performs processing related to encoding of atlas information (including priority control information). For example, the atlas information encoder 351 may acquire atlas information supplied from the displacement vector generator 324. This atlas information includes priority control information generated by applying Method 1. The atlas information encoder 351 may also encode the acquired atlas information using a predetermined encoding method to generate encoded data of the atlas information. The atlas information encoder 351 may also supply the generated encoded data of the atlas information to the multiplexer 358.

[0120] The base mesh encoding unit 352 performs processing related to encoding of the base mesh. For example, the base mesh encoding unit 352 may acquire a base mesh supplied from the displacement vector generation unit 324. The base mesh encoding unit 352 may acquire atlas information supplied from the displacement vector generation unit 324. The base mesh encoding unit 352 may quantize the acquired base mesh and encode it using a predetermined encoding method (e.g., Draco) to generate encoded data of the base mesh. In this case, the base mesh encoding unit 352 may encode the base mesh based on priority control information included in the acquired atlas information. The base mesh encoding unit 352 may supply the generated encoded data of the base mesh to the displacement vector correction unit 353. The base mesh encoding unit 352 may also supply the generated encoded data of the base mesh to the multiplexing unit 358.

[0121] The displacement vector correction unit 353 performs processing related to the correction of the displacement vector. For example, the displacement vector correction unit 353 may acquire a base mesh and a displacement vector supplied from the displacement vector generation unit 324. Alternatively, the displacement vector correction unit 353 may acquire encoded data of the base mesh supplied from the base mesh encoding unit 352. The displacement vector correction unit 353 may correct the displacement vector based on this information. For example, the displacement vector correction unit 353 may decode the acquired encoded data of the base mesh, compare the base mesh before and after encoding to determine encoding distortion of the base mesh, and correct the displacement vector in accordance with the encoding distortion. The displacement vector correction unit 353 may supply the corrected displacement vector to the displacement vector encoding unit 354. Alternatively, the displacement vector correction unit 353 may dequantize the decoded base mesh and supply it to the mesh reconstruction unit 355.

[0122] The displacement vector encoding unit 354 performs processing related to encoding of displacement vectors. For example, the displacement vector encoding unit 354 may acquire displacement vectors supplied from the displacement vector correction unit 353. The displacement vector encoding unit 354 may also generate a displacement map by wavelet transforming the displacement vectors, quantizing them, and packing them into a two-dimensional region. The displacement vector encoding unit 354 may also generate a displacement video in which the displacement map is used as frame images. In other words, the displacement video is a moving image in which the frame images are a displacement map, which is a two-dimensional region in which displacement vectors are packed. The displacement vector encoding unit 354 may also encode the generated displacement video using a predetermined encoding method for 2D moving images to generate encoded data of displacement vectors (displacement video). The displacement vector encoding unit 354 may also supply the encoded data of the displacement vectors generated in this manner to the multiplexing unit 358. The displacement vector encoding unit 354 may also decode the generated encoded data, unpack the displacement vectors from the displacement map, and dequantize the displacement vectors. The displacement vector encoding unit 354 may supply the dequantized displacement vector to the mesh reconstruction unit 355 .

[0123] The displacement vector encoding unit 354 may arithmetically encode the displacement vector to generate encoded data of the displacement vector. In this case, the displacement vector encoding unit 354 may arithmetically decode the encoded data to generate a displacement vector and supply the generated displacement vector to the mesh reconstruction unit 355.

[0124] In addition, the displacement vector encoding unit 354 may acquire atlas information (including priority control information) supplied from the displacement vector generation unit 324, and perform encoding of the displacement vector based on the priority control information contained in the acquired atlas information.

[0125] The mesh reconstruction unit 355 performs processing related to mesh reconstruction. For example, the mesh reconstruction unit 355 may acquire a base mesh supplied from the displacement vector correction unit 353. The mesh reconstruction unit 355 may also acquire a displacement vector supplied from the displacement vector encoding unit 354. The mesh reconstruction unit 355 may use these to reconstruct a mesh. The mesh reconstruction unit 355 may supply the reconstructed mesh to the attribute map conversion unit 356.

[0126] The attribute map conversion unit 356 performs processing related to attribute map conversion. For example, the attribute map conversion unit 356 may acquire a reconstructed mesh supplied from the mesh reconstruction unit 355. Alternatively, the attribute map conversion unit 356 may acquire atlas information supplied from the displacement vector generation unit 324. Alternatively, the attribute map conversion unit 356 may acquire an original mesh and an attribute map input to the encoding device 300. The attribute map conversion unit 356 may convert the acquired attribute map based on other acquired information. For example, the attribute map conversion unit 356 may convert the attribute map based on the atlas information, the original mesh, or the like so that it corresponds to the reconstructed mesh. In other words, the attribute map conversion unit 356 can be said to generate a converted attribute map. Therefore, the attribute map conversion unit 356 can also be said to be an attribute map generation unit. The attribute map conversion unit 356 may supply the converted attribute map to the attribute encoding unit 357 .

[0127] The attribute encoding unit 357 performs processing related to encoding of attributes. For example, the attribute encoding unit 357 may acquire an attribute map supplied from the attribute map conversion unit 356. The attribute encoding unit 357 may also generate attribute video using the acquired attribute map as frame images. The attribute encoding unit 357 may also encode the generated attribute video using a predetermined encoding method for 2D video to generate encoded data of attributes. The attribute encoding unit 357 may also supply the generated encoded data of attributes to the multiplexing unit 358.

[0128] The multiplexing unit 358 performs processing related to multiplexing of encoded data (substreams). For example, the multiplexing unit 358 may acquire encoded data of atlas information supplied from the atlas information encoding unit 351. Alternatively, the multiplexing unit 358 may acquire encoded data of base meshes supplied from the base mesh encoding unit 352. Alternatively, the multiplexing unit 358 may acquire encoded data of displacement vectors supplied from the displacement vector encoding unit 354. Alternatively, the multiplexing unit 358 may acquire encoded data of attributes supplied from the attribute encoding unit 357. The multiplexing unit 358 may multiplex these pieces of encoded data as substreams to generate a V-DMC bitstream. Therefore, the multiplexing unit 358 can also be referred to as a bitstream generation unit (or V-DMC bitstream generation unit). The multiplexing unit 358 may output the generated V-DMC bitstream to the outside of the encoding device 300. For example, the multiplexing unit 358 may supply the V-DMC bitstream to a decoding device 400 (described later). Therefore, the multiplexing unit 358 can also be said to be a supply unit (providing unit) for the V-DMC bitstream.

[0129] That is, in the encoding device 300 (first information processing device), the priority control information generator 323 applies the above-described method 1 to generate priority control information, which is information regarding the priority of processing related to mesh reconstruction. The V-DMC encoding unit 312 applies the above-described method 1 to encode the base mesh, displacement vector, and priority control information, and generate a bitstream.

[0130] With this configuration, the encoding device 300 can use the generated priority control information to control which area of ​​the mesh the decoding device will prioritize for reconstruction. In other words, the encoding device 300 can suppress a decrease in subjective quality and suppress an increase in the processing load related to reconstruction.

[0131] <Flow of Encoding Process> An example of the flow of the encoding process executed by the encoding device 300 will be described with reference to the flowchart of FIG.

[0132] When the encoding process starts, in step S301, the base mesh generation unit 321 of the encoding device 300 decimates the original mesh to be encoded and generates a base mesh.

[0133] In step S302, the atlas information generating unit 322 generates atlas information for the base mesh.

[0134] In step S303, the priority control information generating unit 323 applies method 1 to generate priority control information, which is information regarding the priority of processing related to the reconstruction of the target region of the mesh.

[0135] In step S304, the displacement vector generating unit 324 generates a displacement vector.

[0136] In step S305, the V-DMC encoding unit 312 performs a V-DMC encoding process, applying Method 1 to encode the V-DMC data including the priority control information generated as described above, to generate a V-DMC bitstream.

[0137] The encoding process ends when the process of step S305 ends. The encoding device 300 executes such encoding process for each frame of the original mesh.

[0138] <Flow of V-DMC Encoding Process> Next, an example of the flow of the V-DMC encoding process executed in step S305 of FIG. 17 will be described with reference to the flowchart of FIG.

[0139] When the V-DMC encoding process starts, the atlas information encoding unit 351 encodes the atlas information including the priority control information in step S321.

[0140] In step S322, the base mesh encoding unit 352 encodes the base mesh. At this time, the base mesh encoding unit 352 may encode the base mesh based on the priority control information.

[0141] In step S323, the displacement vector correction unit 353 corrects the displacement vector.

[0142] In step S324, the displacement vector encoding unit 354 encodes the corrected displacement vector. For example, the displacement vector encoding unit 354 may pack the displacement vector into a displacement video and encode it using a 2D encoding method. Alternatively, the displacement vector encoding unit 354 may perform arithmetic encoding on the displacement vector. In this case, the displacement vector encoding unit 354 may encode the displacement vector based on priority control information.

[0143] In step S325, the mesh reconstructing unit 355 reconstructs the mesh.

[0144] In step S326, the attribute map conversion unit 356 converts the attribute map.

[0145] In step S327, the attribute encoding unit 357 encodes the attribute video using the attribute map as a frame image.

[0146] In step S328, the multiplexing unit 358 multiplexes the encoded data of the atlas information (including priority control information), the encoded data of the base mesh, the encoded data of the displacement vector, and the encoded data of the attributes to generate a V-DMC bitstream.

[0147] When the process of step S328 ends, the V-DMC encoding process ends, and the process returns to FIG.

[0148] By performing each process as described above, the encoding device 300 can control, using the generated priority control information, which area of ​​the mesh the decoding device will prioritize for reconstruction. In other words, the encoding device 300 can suppress a decrease in subjective quality and suppress an increase in the processing load related to reconstruction.

[0149] 7. Second embodiment Decoding device The present technology can be applied to a decoding device that decodes encoded data of a mesh. Fig. 19 is a block diagram showing an example of the configuration of a decoding device that is one aspect of an information processing device to which the present technology is applied. The decoding device 400 (second information processing device) shown in Fig. 19 is a device that decodes, for example, encoded data of a mesh generated in the encoding device 300 (Fig. 15) (a V-DMC bitstream generated by the multiplexing unit 358 (Fig. 16)) and reconstructs a decoded mesh.

[0150] Fig. 19 shows the main processing units, data flows, etc., but does not necessarily include all of them. In other words, in the decoding device 400, there may be processing units that are not shown as blocks in Fig. 19, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 19.

[0151] The decoding device 400 decodes coded data of a mesh that has been coded using a method essentially similar to the V-DMC method described in the aforementioned non-patent document, except that the present technology is applied, and reconstructs the decoded mesh. For example, the decoding device 400 obtains a V-DMC bitstream. This V-DMC bitstream may be generated by, for example, the coding device 300. As part of the reconstruction process, the decoding device 400 decodes the V-DMC bitstream and reconstructs a mesh (also referred to as a decoded mesh). The decoding device 400 also applies a texture to the decoded mesh, generates a display image for displaying the decoded mesh, and outputs the display image to an external device. For example, the decoding device 400 supplies the display image to an external display device for display.

[0152] As shown in Figure 19, the decoding device 400 (second information processing device) has a demultiplexing unit 411, an atlas information decoding unit 412, a base mesh decoding unit 413, a subdivision unit 414, a displacement vector decoding unit 415, a displacement vector application unit 416, an attribute decoding unit 417, an attribute application unit 418, and a display processing unit 419.

[0153] The demultiplexing unit 411 performs demultiplexing processing. For example, the demultiplexing unit 411 may acquire a V-DMC bitstream to be decoded and supplied to the decoding device 400. The demultiplexing unit 411 may also demultiplex the acquired V-DMC bitstream to extract coded data of atlas information, coded data of base meshes, coded data of displacement vectors, and coded data of attributes. Therefore, the demultiplexing unit 411 can also be considered an acquirer of a V-DMC bitstream or various information contained in the V-DMC bitstream. The demultiplexing unit 411 may supply the coded data of the extracted atlas information to the atlas information decoding unit 412. The demultiplexing unit 411 may also supply the coded data of the extracted base meshes to the base mesh decoding unit 413. The demultiplexing unit 411 may also supply the coded data of the extracted displacement vectors to the displacement vector decoding unit 415. Furthermore, the demultiplexing unit 411 may supply the coded data of the extracted attributes to the attribute decoding unit 417 .

[0154] The atlas information decoding unit 412 performs processing related to decoding of the atlas information. For example, the atlas information decoding unit 412 may acquire encoded data of the atlas information supplied from the demultiplexing unit 411. The atlas information decoding unit 412 may also decode the acquired encoded data of the atlas information to generate (restore) the atlas information. This atlas information includes priority control information. Therefore, the atlas information decoding unit 412 can also be referred to as a priority control information decoding unit. The atlas information decoding unit 412 may supply the generated atlas information to the base mesh decoding unit 413. The atlas information decoding unit 412 may supply the generated atlas information (including the priority control information) to the subdivision unit 414. The atlas information decoding unit 412 may supply the generated atlas information (including the priority control information) to the displacement vector decoding unit 415. The atlas information decoding unit 412 may supply the generated atlas information (including the priority control information) to the displacement vector application unit 416. The atlas information decoding unit 412 may supply the generated atlas information to the attribute decoding unit 417. The atlas information decoding unit 412 may supply the generated atlas information to the attribute application unit 418.

[0155] The base mesh decoding unit 413 performs processing related to decoding of the base mesh. For example, the base mesh decoding unit 413 may acquire coded data of the base mesh supplied from the demultiplexing unit 411. The base mesh decoding unit 413 may also decode the acquired coded data of the base mesh using a predetermined decoding method (e.g., Draco) to generate (restore) a base mesh (e.g., a vertex list, a triangle list, etc.). In this case, the base mesh decoding unit 413 may acquire atlas information supplied from the atlas information decoding unit 412 and decode the coded data of the base mesh based on the atlas information. The base mesh decoding unit 413 may also supply the generated base mesh to the subdivision unit 414.

[0156] The subdivision unit 414 performs processing related to subdivision of triangles of a base mesh. For example, the subdivision unit 414 may acquire a base mesh supplied from the base mesh decoding unit 413. The subdivision unit 414 may subdivide the base mesh (triangles) to generate division points. In this case, the subdivision unit 414 may acquire atlas information (including priority control information) supplied from the atlas information decoding unit 412 and subdivide the base mesh based on the priority control information. For example, the subdivision unit 414 may preferentially subdivide the base mesh in a priority region specified by the priority control information. The subdivision unit 414 may supply the subdivided base mesh to the displacement vector application unit 416.

[0157] The displacement vector decoding unit 415 performs processing related to the decoding of displacement vectors. For example, the displacement vector decoding unit 415 may acquire encoded data of displacement vectors (i.e., a displacement bitstream) supplied from the demultiplexing unit 411. The displacement vector decoding unit 415 may decode the encoded data of the displacement vectors to generate (restore) displacement vectors. For example, if the displacement vectors are encoded as displacement video, the displacement vector decoding unit 415 may decode the encoded data of the displacement vectors using a predetermined decoding method for 2D video, generate (restore) the displacement video, and unpack the displacement vectors from a displacement map, which is a frame image of the displacement video. Furthermore, if the displacement video is arithmetically coded, the displacement vector decoding unit 415 arithmetically decodes the encoded data of the displacement vectors to generate displacement vectors. In this case, the displacement vector decoding unit 415 may acquire atlas information (including priority control information) supplied from the atlas information decoding unit 412 and decode the displacement vectors based on the priority control information. For example, the displacement vector decoding unit 415 may preferentially decode the displacement vector of the priority region specified by the priority control information, and may supply the displacement vector obtained in this manner to the displacement vector application unit 416.

[0158] The displacement vector application unit 416 performs processing related to application of displacement vectors to the subdivided base mesh. For example, the displacement vector application unit 416 may acquire the subdivided base mesh supplied from the subdivision unit 414. The displacement vector application unit 416 may acquire displacement vectors supplied from the displacement vector decoding unit 415. The displacement vector application unit 416 may apply displacement vectors to the vertices of the subdivided base mesh. In other words, the displacement vector application unit 416 may generate a decoded mesh. In this case, the displacement vector application unit 416 may acquire atlas information (including priority control information) supplied from the atlas information decoding unit 412 and apply displacement vectors to the vertices of the subdivided base mesh based on the priority control information. For example, the displacement vector application unit 416 may preferentially apply displacement vectors to vertices of the subdivided base mesh in a priority region specified by the priority control information. The displacement vector application unit 416 may supply the decoded mesh generated in this manner to the attribute application unit 418.

[0159] The attribute decoding unit 417 executes processing related to attribute decoding. For example, the attribute decoding unit 417 may acquire coded attribute data supplied from the demultiplexing unit 411. The attribute decoding unit 417 may also decode the acquired coded attribute data using a predetermined decoding method for 2D video to generate (restore) attribute video. In this case, the attribute decoding unit 417 may acquire atlas information supplied from the atlas information decoding unit 412 and decode attributes based on the atlas information. The attribute decoding unit 417 may also supply an attribute map, which is a frame image of the generated attribute video, to the attribute application unit 418.

[0160] The attribute application unit 418 performs processing related to the application of attributes to the decoded mesh. For example, the attribute application unit 418 may acquire the decoded mesh supplied from the displacement vector application unit 416. The attribute application unit 418 may acquire an attribute map supplied from the attribute decoding unit 417. The attribute application unit 418 may apply attributes of the attribute map to the decoded mesh. In this case, the attribute application unit 418 may acquire atlas information supplied from the atlas information decoding unit 412 and apply attributes to the decoded mesh based on the atlas information. The attribute application unit 418 may supply the decoded mesh to which the attributes have been applied in this manner to the display processing unit 419.

[0161] The display processing unit 419 performs processing related to mesh display. For example, the display processing unit 419 may acquire a decoded mesh to which attributes have been applied, supplied from the attribute application unit 418. The display processing unit 419 may render the acquired decoded mesh and generate a display image for displaying the decoded mesh. The display processing unit 419 may then supply the generated display image to an external device outside the decoding device 400, and display the display image on another device or the like.

[0162] In the decoding device 400 (second information processing device), the atlas information decoding unit 412 applies Method 1 to decode a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction. The subdivision unit 414 subdivides the base mesh as part of the reconstruction processing. The displacement vector application unit 416 applies displacement vectors to the subdivided base mesh in accordance with the priority order based on the priority control information as part of the reconstruction processing.

[0163] With this configuration, the decoding device 400 can control which area of ​​the mesh is to be preferentially reconstructed in accordance with the generated priority control information. In other words, the decoding device 400 can suppress a decrease in subjective quality and suppress an increase in the processing load related to reconstruction.

[0164] <Flow of Decoding Process> An example of the flow of the decoding process executed by the decoding device 400 will be described with reference to the flowchart of FIG.

[0165] When the decoding process starts, the demultiplexing unit 411 of the decoding device 400 demultiplexes the V-DMC bitstream in step S401.

[0166] In step S402, the atlas information decoding unit 412 applies Method 1 to decode the encoded data of the atlas information and generate (restore) atlas information including priority control information. That is, the atlas information decoding unit 412 decodes a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction.

[0167] In step S403, the base mesh decoding unit 413 decodes the base meshes of the entire region.

[0168] In step S404, the subdivision unit 414, the displacement vector decoding unit 415, and the displacement vector application unit 416 apply method 1 to perform a decoded mesh reconstruction process and reconstruct the decoded mesh in accordance with the priority control information. That is, the subdivision unit 414, the displacement vector decoding unit 415, and the displacement vector application unit 416 perform reconstruction-related processes, including subdivision of the base mesh, decoding of the displacement bitstream, and application of the displacement vector, based on the priority control information, and reconstruct the priority area of ​​the decoded mesh in preference to the non-priority area.

[0169] In step S405, the attribute decoding unit 417 decodes the attributes of all the regions. That is, the attribute decoding unit 417 decodes the coded data of the attributes of all the regions and generates (restores) the attributes of all the regions.

[0170] In step S406, the attribute application unit 418 applies the attributes of the entire region to the decoded mesh.

[0171] In step S407, the display processing unit 419 renders the decoded mesh to which the attributes have been applied to generate a display image.

[0172] When the process of step S407 is completed, the decoding process is completed. The decoding device 400 executes such a decoding process for each frame of the original mesh.

[0173] <Flow 1 of decoding mesh reconstruction process> Next, an example of the flow of the decoding mesh reconstruction process executed in step S404 of Fig. 20 will be described with reference to the flowcharts of Fig. 21 and Fig. 22. Note that here, a case will be described in which the priority region of the base mesh is preferentially subdivided based on priority control information.

[0174] When the decoding mesh reconstruction process is started, the subdivision unit 414, the displacement vector decoding unit 415, and the displacement vector application unit 416 determine whether or not a priority region exists (i.e., whether or not a priority region has been specified) based on the priority control information in step S421. If it is determined that a priority region exists, the process proceeds to step S422.

[0175] In step S422, the subdivision unit 414 subdivides the priority region of the base mesh.

[0176] In step S423, the disparity vector decoding unit 415 decodes the disparity vector corresponding to the priority region. That is, the disparity vector decoding unit 415 decodes the priority bitstream and generates (restores) the disparity vector corresponding to the priority region.

[0177] In step S424, the displacement vector applicator 416 applies the displacement vector corresponding to the priority region to the priority region (i.e., the subdivided portion) of the base mesh.

[0178] In step S425, the subdivision unit 414, the displacement vector decoding unit 415, and the displacement vector application unit 416 determine whether or not a predetermined parameter related to the reconstruction process has reached a predetermined limit. If it is determined that the predetermined parameter related to the reconstruction process has not reached the limit, the process proceeds to step S426. For example, if it is determined that the number of faces in the decoded mesh has not reached the upper limit, the process proceeds to step S426.

[0179] In step S426, the subdivision unit 414 subdivides the non-priority regions of the base mesh.

[0180] In step S427, the disparity vector decoding unit 415 decodes the disparity vector corresponding to the non-priority region. That is, the disparity vector decoding unit 415 decodes the non-priority bitstream and generates (restores) the disparity vector corresponding to the non-priority region.

[0181] In step S428, the displacement vector applicator 416 applies the displacement vector corresponding to the non-priority region (i.e., the subdivided portion) of the base mesh.

[0182] When the processing of step S428 is completed, the decoded mesh reconstruction processing is completed. Also, if it is determined in step S425 that a predetermined parameter related to the reconstruction processing has reached its limit (for example, the number of faces of the decoded mesh has reached its upper limit), the decoded mesh reconstruction processing is completed. When the decoded mesh reconstruction processing is completed, the processing returns to FIG. 20.

[0183] Also, if it is determined in step S421 that no priority area exists, the process proceeds to FIG.

[0184] In step S441 of FIG. 22, the subdivision unit 414 subdivides the base mesh of the entire region.

[0185] In step S442, the displacement vector decoding unit 415 decodes the displacement vectors of all regions.

[0186] In step S443, the displacement vector application unit 416 applies the displacement vector to the base mesh of the entire subdivided region.

[0187] When the process of step S443 is completed, the process returns to Fig. 21, and the decoded mesh reconstruction process is completed. When the decoded mesh reconstruction process is completed, the process returns to Fig. 20.

[0188] By performing each process as described above, the decoding device 400 can control which area of ​​the mesh is to be preferentially reconstructed in accordance with the priority control information. In other words, the decoding device 400 can suppress a decrease in subjective quality and suppress an increase in the processing load related to reconstruction.

[0189] In this decoding mesh reconstruction processing, if a predetermined parameter related to the reconstruction processing reaches a limit during each of the processes from step S422 to step S424 in Fig. 21, the decoding mesh reconstruction processing may be terminated. Similarly, if a predetermined parameter related to the reconstruction processing reaches a limit during each of the processes from step S426 to step S428 in Fig. 21, the decoding mesh reconstruction processing may be terminated. Similarly, if a predetermined parameter related to the reconstruction processing reaches a limit during each of the processes from step S441 to step S443 in Fig. 22, the decoding mesh reconstruction processing may be terminated.

[0190] In this decoding mesh reconstruction process, the decoding of the displacement vectors may be performed for the entire region. For example, before the process of step S421 described above, the displacement vector decoding unit 415 may decode the displacement bitstreams for the entire region and generate (restore) the displacement vectors corresponding to the entire region.

[0191] <Flow 2 of Decoding Mesh Reconstruction Processing> Next, another example of the flow of the decoding mesh reconstruction processing executed in step S404 of Fig. 20 will be described with reference to the flowcharts of Fig. 23 and Fig. 24. Here, a case will be described in which the base meshes of the entire region are subdivided, and displacement vectors corresponding to the priority region are preferentially decoded based on the priority control information.

[0192] When the decoded mesh reconstruction process is started, the subdivision unit 414 subdivides the entire region of the base mesh in step S461.

[0193] In step S462, the subdivision unit 414, the displacement vector decoding unit 415, and the displacement vector application unit 416 determine whether or not a priority area exists (i.e., whether or not a priority area has been specified) based on the priority control information. If it is determined that a priority area exists, the process proceeds to step S463.

[0194] In step S463, the disparity vector decoding unit 415 decodes the disparity vector corresponding to the priority region. That is, the disparity vector decoding unit 415 decodes the priority bitstream and generates (restores) the disparity vector corresponding to the priority region.

[0195] In step S464, the displacement vector applicator 416 applies the displacement vector corresponding to the priority region to the priority region (i.e., the subdivided portion) of the base mesh.

[0196] In step S465, the subdivision unit 414, the displacement vector decoding unit 415, and the displacement vector application unit 416 determine whether or not a predetermined parameter related to the reconstruction process has reached a predetermined limit. If it is determined that the predetermined parameter related to the reconstruction process has not reached the limit, the process proceeds to step S466. For example, if it is determined that the number of faces in the decoded mesh has not reached the upper limit, the process proceeds to step S466.

[0197] In step S466, the disparity vector decoding unit 415 decodes the disparity vector corresponding to the non-priority region. That is, the disparity vector decoding unit 415 decodes the non-priority bitstream and generates (restores) the disparity vector corresponding to the non-priority region.

[0198] In step S467, the displacement vector application unit 416 applies the displacement vector corresponding to the non-priority region (i.e., the subdivided portion) of the base mesh.

[0199] When the processing of step S467 is completed, the decoded mesh reconstruction processing is completed. Also, if it is determined in step S465 that a predetermined parameter related to the reconstruction processing has reached its limit (for example, the number of faces of the decoded mesh has reached its upper limit), the decoded mesh reconstruction processing is completed. When the decoded mesh reconstruction processing is completed, the processing returns to FIG. 20.

[0200] Also, if it is determined in step S462 that no priority area exists, the process proceeds to FIG.

[0201] In step S481 of FIG. 24, the displacement vector decoding unit 415 decodes the displacement vectors of all regions.

[0202] In step S482, the displacement vector application unit 416 applies the displacement vector of the entire region to the entire region of the subdivided base mesh.

[0203] When the process of step S482 is completed, the process returns to Fig. 23, and the decoded mesh reconstruction process is completed. When the decoded mesh reconstruction process is completed, the process returns to Fig. 20.

[0204] By performing each process as described above, the decoding device 400 can control which area of ​​the mesh is to be preferentially reconstructed in accordance with the priority control information. In other words, the decoding device 400 can suppress a decrease in subjective quality and suppress an increase in the processing load related to reconstruction.

[0205] 8. Third Embodiment Encoding Device Fig. 25 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology is applied. The encoding device 500 (third information processing device) shown in Fig. 25 is a device that encodes a mesh and generates a bitstream thereof. Therefore, the encoding device 500 can also be said to be a bitstream generation device that generates a bitstream.

[0206] Fig. 25 shows the main processing units, data flows, etc., but is not limited to all that is shown in Fig. 25. In other words, in encoding device 500, there may be processing units that are not shown as blocks in Fig. 25, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 25.

[0207] The encoding device 500 encodes a mesh using a method essentially similar to the V-DMC method described in the aforementioned non-patent document, except that the present technology is applied. For example, the encoding device 500 acquires an original mesh to be encoded and an attribute map including a texture corresponding to the original mesh. Note that the original mesh includes not only mesh geometry information but also information indicating the correspondence with the attribute map (e.g., a UV list). The encoding device 500 encodes the original mesh and attribute map using the V-DMC method, generates a V-DMC bitstream, and outputs it.

[0208] 25 , the encoding device 500 (third information processing device) has a preprocessing unit 511 and a V-DMC encoding unit 512. The preprocessing unit 511 performs preprocessing before encoding. As shown in FIG. 25 , the preprocessing unit 511 has a base mesh generation unit 521, an atlas information generation unit 522, a reconstruction control information generation unit 523, and a displacement vector generation unit 524.

[0209] The base mesh generation unit 521 performs processing related to the generation of a base mesh. For example, the base mesh generation unit 521 may obtain an original mesh to be input to the encoding device 500. The base mesh generation unit 521 may also perform decimation processing (thinning out vertices) on the original mesh to generate a base mesh. The base mesh generation unit 521 may supply the generated base mesh together with the original mesh to the atlas information generation unit 522.

[0210] The atlas information generation unit 522 performs processing related to the generation of atlas information corresponding to a base mesh. For example, the atlas information generation unit 522 may acquire a base mesh or an original mesh supplied from the base mesh generation unit 521. The atlas information generation unit 522 may generate atlas information by, for example, UV unwrapping the base mesh. The atlas information generation unit 522 may exchange information with the reconstruction control information generation unit 523. For example, the atlas information generation unit 522 may acquire reconstruction control information supplied from the reconstruction control information generation unit 523. The atlas information generation unit 522 may include the reconstruction control information in the atlas information. The atlas information generation unit 522 may supply the generated atlas information (including the reconstruction control information) to the displacement vector generation unit 524 together with the base mesh, etc.

[0211] The reconstruction control information generating unit 523 executes processing related to the generation of the reconstruction control information. For example, the reconstruction control information generating unit 523 may generate the reconstruction control information by applying the above-described method 2. Furthermore, when generating the reconstruction control information, the reconstruction control information generating unit 523 may acquire information such as a base mesh or an original mesh supplied from the atlas information generating unit 522 and use the base mesh or the original mesh. The reconstruction control information generating unit 523 may supply the generated reconstruction control information to the atlas information generating unit 522.

[0212] As shown in FIG. 25 , the reconstruction control information generator 523 may include a target region designation information generator 531 and a control information generator 532 .

[0213] The target region designation information generation unit 531 executes processing related to the generation of target region designation information. For example, the target region designation information generation unit 531 may generate the target region designation information by applying the above-described method 2. Furthermore, when generating the target region designation information, the target region designation information generation unit 531 may obtain information such as a base mesh or an original mesh supplied from the atlas information generation unit 522 and use the base mesh or the original mesh.

[0214] The control information generating unit 532 executes processing related to the generation of control information. For example, the control information generating unit 532 may generate the control information by applying the above-described method 2. Furthermore, when generating the control information, the control information generating unit 532 may obtain information such as a base mesh or an original mesh supplied from the atlas information generating unit 522 and use the base mesh or the original mesh.

[0215] The reconstruction control information generating unit 523 may supply the target region designation information and control information thus generated to the atlas information generating unit 522 as reconstruction control information.

[0216] The displacement vector generation unit 524 performs processing related to the generation of displacement vectors. For example, the displacement vector generation unit 524 may acquire a base mesh, atlas information, etc. supplied from the atlas information generation unit 522. Alternatively, the displacement vector generation unit 524 may acquire an original mesh input to the encoding device 500. Alternatively, the displacement vector generation unit 524 may use this information to generate displacement vectors that displace vertices of the subdivided base mesh. The displacement vector generation unit 524 may supply the generated displacement vectors to the V-DMC encoding unit 512 together with the base mesh, atlas information (including reconstruction control information), etc.

[0217] The V-DMC encoder 512 performs processing related to encoding of V-DMC data. For example, the V-DMC encoder 512 may acquire an original mesh input to the encoding device 500. The V-DMC encoder 512 may also acquire a base mesh, a displacement vector, atlas information (including reconstruction control information), and the like, supplied from the displacement vector generator 524. The V-DMC encoder 512 may also acquire an attribute map input to the encoding device 500. The V-DMC encoder 512 may use this information to encode the atlas information (including reconstruction control information), the base mesh, the displacement vector, and the attribute map, respectively, and generate the respective encoded data. Therefore, the V-DMC encoder 512 can also be referred to as an encoder. The V-DMC encoder 512 may also multiplex these encoded data as substreams to generate a single bitstream. This bitstream is also referred to as a V-DMC bitstream. Therefore, the V-DMC encoder 512 can also be referred to as a bitstream generator. The V-DMC encoding unit 512 may output the generated V-DMC bitstream to the outside of the encoding device 500 .

[0218] <V-DMC Encoder> Fig. 26 is a block diagram showing an example of the main configuration of the V-DMC encoder 512. Note that Fig. 26 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, the V-DMC encoder 512 may include processing units that are not shown as blocks in Fig. 26, and may include processing and data flows that are not shown as arrows or the like in Fig. 26.

[0219] As shown in FIG. 26, the V-DMC encoding unit 512 has an atlas information encoding unit 551, a base mesh encoding unit 552, a displacement vector correction unit 553, a displacement vector encoding unit 554, a mesh reconstruction unit 555, an attribute map conversion unit 556, an attribute encoding unit 557, and a multiplexing unit 558.

[0220] The atlas information encoding unit 551 performs processing related to encoding of atlas information (including reconstruction control information). For example, the atlas information encoding unit 551 may acquire atlas information supplied from the displacement vector generation unit 524. This atlas information includes reconstruction control information generated by applying method 2. The atlas information encoding unit 551 may also encode the acquired atlas information using a predetermined encoding method to generate encoded data of the atlas information. The atlas information encoding unit 551 may also supply the generated encoded data of the atlas information to the multiplexing unit 558.

[0221] The base mesh encoding unit 552 performs processing related to encoding of the base mesh. For example, the base mesh encoding unit 552 may acquire a base mesh supplied from the displacement vector generation unit 524. The base mesh encoding unit 552 may acquire atlas information supplied from the displacement vector generation unit 524. The base mesh encoding unit 552 may quantize the acquired base mesh and encode it using a predetermined encoding method (e.g., Draco) to generate encoded data of the base mesh. In this case, the base mesh encoding unit 552 may encode the base mesh based on reconstruction control information included in the acquired atlas information. For example, the base mesh encoding unit 552 may encode the base mesh using an encoding method (e.g., intra, inter, etc.) specified by the reconstruction control information. The base mesh encoding unit 552 may supply the generated encoded data of the base mesh to the displacement vector correction unit 553. The base mesh encoding unit 552 may also supply the generated encoded data of the base mesh to the multiplexing unit 558.

[0222] The displacement vector correction unit 553 performs processing related to the correction of the displacement vector. For example, the displacement vector correction unit 553 may acquire a base mesh and a displacement vector supplied from the displacement vector generation unit 524. Alternatively, the displacement vector correction unit 553 may acquire encoded data of the base mesh supplied from the base mesh encoding unit 552. The displacement vector correction unit 553 may correct the displacement vector based on this information. For example, the displacement vector correction unit 553 may decode the acquired encoded data of the base mesh, compare the base mesh before and after encoding to determine encoding distortion of the base mesh, and correct the displacement vector in accordance with the encoding distortion. The displacement vector correction unit 553 may supply the corrected displacement vector to the displacement vector encoding unit 554. Alternatively, the displacement vector correction unit 553 may dequantize the decoded base mesh and supply it to the mesh reconstruction unit 555.

[0223] The displacement vector encoding unit 554 performs processing related to encoding of displacement vectors. For example, the displacement vector encoding unit 554 may acquire displacement vectors supplied from the displacement vector correction unit 553. The displacement vector encoding unit 554 may also generate a displacement map by wavelet transforming the displacement vectors, quantizing them, and packing them into a two-dimensional region. The displacement vector encoding unit 554 may also generate a displacement video in which the displacement map is used as frame images. In other words, the displacement video is a moving image in which the frame images are displacement maps, which are two-dimensional regions in which displacement vectors are packed. The displacement vector encoding unit 554 may also encode the generated displacement video using a predetermined encoding method for 2D moving images to generate encoded data of displacement vectors (displacement video). The displacement vector encoding unit 554 may also supply the encoded data of the displacement vectors generated in this manner to the multiplexing unit 558. The displacement vector encoding unit 554 may also decode the generated encoded data, unpack the displacement vectors from the displacement map, and dequantize the displacement vectors. The displacement vector encoding unit 554 may supply the dequantized displacement vector to the mesh reconstruction unit 555 .

[0224] The displacement vector encoding unit 554 may arithmetically encode the displacement vector to generate encoded data of the displacement vector. In this case, the displacement vector encoding unit 554 may arithmetically decode the encoded data to generate a displacement vector and supply the generated displacement vector to the mesh reconstruction unit 555.

[0225] In addition, the displacement vector encoding unit 554 may acquire atlas information (including reconstruction control information) supplied from the displacement vector generation unit 524, and perform encoding of the displacement vector based on the reconstruction control information contained in the acquired atlas information.

[0226] The mesh reconstruction unit 555 performs processing related to mesh reconstruction. For example, the mesh reconstruction unit 555 may acquire a base mesh supplied from the displacement vector correction unit 553. The mesh reconstruction unit 555 may also acquire a displacement vector supplied from the displacement vector encoding unit 554. The mesh reconstruction unit 555 may reconstruct a mesh using these. The mesh reconstruction unit 555 may supply the reconstructed mesh to the attribute map conversion unit 556.

[0227] The attribute map conversion unit 556 performs processing related to attribute map conversion. For example, the attribute map conversion unit 556 may acquire a reconstructed mesh supplied from the mesh reconstruction unit 555. The attribute map conversion unit 556 may also acquire atlas information supplied from the displacement vector generation unit 524. The attribute map conversion unit 556 may also acquire an original mesh and an attribute map input to the encoding device 500. The attribute map conversion unit 556 may convert the acquired attribute map based on other acquired information. For example, the attribute map conversion unit 556 may convert the attribute map based on the atlas information, the original mesh, etc., so that it corresponds to the reconstructed mesh. In other words, the attribute map conversion unit 556 can be said to generate a converted attribute map. Therefore, the attribute map conversion unit 556 can also be said to be an attribute map generation unit. The attribute map conversion unit 556 may supply the converted attribute map to the attribute encoding unit 557 .

[0228] The attribute encoding unit 557 performs processing related to encoding of attributes. For example, the attribute encoding unit 557 may acquire an attribute map supplied from the attribute map conversion unit 556. The attribute encoding unit 557 may also generate attribute video using the acquired attribute map as frame images. The attribute encoding unit 557 may also encode the generated attribute video using a predetermined encoding method for 2D video to generate encoded data of attributes. The attribute encoding unit 557 may also supply the generated encoded data of attributes to the multiplexing unit 558.

[0229] The multiplexing unit 558 performs processing related to multiplexing of encoded data (substreams). For example, the multiplexing unit 558 may acquire encoded data of atlas information supplied from the atlas information encoding unit 551. Alternatively, the multiplexing unit 558 may acquire encoded data of base meshes supplied from the base mesh encoding unit 552. Alternatively, the multiplexing unit 558 may acquire encoded data of displacement vectors supplied from the displacement vector encoding unit 554. Alternatively, the multiplexing unit 558 may acquire encoded data of attributes supplied from the attribute encoding unit 557. The multiplexing unit 558 may multiplex these pieces of encoded data as substreams to generate a V-DMC bitstream. Therefore, the multiplexing unit 558 can also be referred to as a bitstream generation unit (or V-DMC bitstream generation unit). The multiplexing unit 558 may output the generated V-DMC bitstream to the outside of the encoding device 500. For example, the multiplexing unit 558 may supply the V-DMC bitstream to a decoding device 600 (described later). Therefore, the multiplexing unit 558 can also be said to be a supply unit (providing unit) for the V-DMC bitstream.

[0230] That is, in the encoding device 500 (third information processing device), the target region designation information generation unit 531 applies the above-described method 1 to generate target region designation information that designates a target region for control of processing related to mesh reconstruction. This target region designation information designates the target region using texture information related to the mesh texture. The control information generation unit 532 applies the above-described method 2 to generate control information that indicates control for the target region. The V-DMC encoding unit 512 applies the above-described method 2 to encode the base mesh, displacement vector, target region designation information, and control information, and generates a bitstream.

[0231] With this configuration, the encoding device 500 can control each process included in the reconstruction-related process using reconstruction control information including target area designation information and control information that uses texture information. Therefore, even if the target area is made smaller to prevent a decrease in the performance of the reconstruction control, the increase in the amount of code can be suppressed compared to the case of sub-mesh units. In other words, the encoding device 500 can prevent a decrease in encoding efficiency and also prevent a decrease in the performance of the reconstruction-related process control.

[0232] <Flow of Encoding Process> An example of the flow of encoding process executed by the encoding device 500 will be described with reference to the flowchart of FIG.

[0233] When the encoding process starts, in step S501, the base mesh generation unit 521 of the encoding device 500 decimates the original mesh to be encoded and generates a base mesh.

[0234] In step S502, the atlas information generating unit 522 generates atlas information for the base mesh.

[0235] In step S503, the target area designation information generating unit 531 applies Method 2 to generate target area designation information that uses texture information.

[0236] In step S504, the control information generating unit 532 applies method 2 to generate control information corresponding to the target area designation information.

[0237] In step S505, the displacement vector generating unit 524 generates a displacement vector.

[0238] In step S506, the V-DMC encoding unit 512 performs a V-DMC encoding process, applying method 2 to encode the V-DMC data including the reconstruction control information (target region designation information and control information) generated as described above, to generate a V-DMC bitstream.

[0239] The encoding process ends when the process of step S506 is completed. The encoding device 500 executes such encoding process for each frame of the original mesh.

[0240] <Flow of V-DMC Encoding Process> Next, an example of the flow of the V-DMC encoding process executed in step S506 of FIG. 27 will be described with reference to the flowchart of FIG.

[0241] When the V-DMC encoding process starts, in step S521, the atlas information encoding unit 551 encodes atlas information including reconstruction control information (target region designation information and control information).

[0242] In step S522, the base mesh encoding unit 552 encodes the base mesh. At this time, the base mesh encoding unit 552 may encode the base mesh based on the reconstruction control information.

[0243] In step S523, the displacement vector correction unit 553 corrects the displacement vector.

[0244] In step S524, the displacement vector encoding unit 554 encodes the corrected displacement vector. For example, the displacement vector encoding unit 554 may pack the displacement vector into a displacement video and encode it using a 2D encoding method. Alternatively, the displacement vector encoding unit 554 may perform arithmetic encoding on the displacement vector. In this case, the displacement vector encoding unit 554 may encode the displacement vector based on the reconstruction control information.

[0245] In step S525, the mesh reconstructing unit 555 reconstructs the mesh.

[0246] In step S526, the attribute map conversion unit 556 converts the attribute map.

[0247] In step S527, the attribute encoding unit 557 encodes the attribute video using the attribute map as a frame image.

[0248] In step S528, the multiplexing unit 558 multiplexes the encoded data of the atlas information (including reconstruction control information), the encoded data of the base mesh, the encoded data of the displacement vector, and the encoded data of the attributes to generate a V-DMC bitstream.

[0249] When the process of step S528 ends, the V-DMC encoding process ends, and the process returns to FIG.

[0250] By performing each process as described above, the encoding device 500 can control each process included in the reconstruction-related process using reconstruction control information including target region designation information and control information that utilizes texture information. Therefore, even if the target region is made smaller to prevent a decrease in the performance of the reconstruction control, the increase in the amount of code can be suppressed compared to the case of sub-mesh units. In other words, the encoding device 500 can prevent a decrease in encoding efficiency and a decrease in the performance of the reconstruction control.

[0251] 9. Fourth Embodiment Decoding Device The present technology can be applied to a decoding device that decodes encoded data of a mesh. Fig. 29 is a block diagram showing an example of the configuration of a decoding device that is one aspect of an information processing device to which the present technology is applied. The decoding device 600 (fourth information processing device) shown in Fig. 29 is a device that decodes, for example, encoded data of a mesh generated in the encoding device 500 (Fig. 25) (a V-DMC bitstream generated by the multiplexing unit 558 (Fig. 26)) and reconstructs a decoded mesh.

[0252] Fig. 29 shows the main processing units, data flows, etc., but does not necessarily include all of them. In other words, in the decoding device 600, there may be processing units that are not shown as blocks in Fig. 29, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 29.

[0253] The decoding device 600 decodes coded data of a mesh that has been coded using a method essentially similar to the V-DMC method described in the aforementioned non-patent document, except that the present technology is applied, and reconstructs the decoded mesh. For example, the decoding device 600 obtains a V-DMC bitstream. This V-DMC bitstream may be generated by, for example, the coding device 500. As part of the reconstruction process, the decoding device 600 decodes the V-DMC bitstream and reconstructs a mesh (also referred to as a decoded mesh). The decoding device 600 also applies a texture to the decoded mesh, generates a display image for displaying the decoded mesh, and outputs the display image to an external device. For example, the decoding device 600 supplies the display image to an external display device for display.

[0254] As shown in Figure 29, the decoding device 600 (fourth information processing device) has a demultiplexing unit 611, an atlas information decoding unit 612, a base mesh decoding unit 613, a subdivision unit 614, a displacement vector decoding unit 615, a displacement vector application unit 616, an attribute decoding unit 617, an attribute application unit 618, and a display processing unit 619.

[0255] The demultiplexing unit 611 performs demultiplexing processing. For example, the demultiplexing unit 611 may acquire a V-DMC bitstream to be decoded and supplied to the decoding device 600. The demultiplexing unit 611 may also demultiplex the acquired V-DMC bitstream to extract coded data of atlas information, coded data of base meshes, coded data of displacement vectors, and coded data of attributes. Therefore, the demultiplexing unit 611 can also be considered an acquirer of a V-DMC bitstream or various information contained in the V-DMC bitstream. The demultiplexing unit 611 may supply the coded data of the extracted atlas information to the atlas information decoding unit 612. The demultiplexing unit 611 may also supply the coded data of the extracted base meshes to the base mesh decoding unit 613. The demultiplexing unit 611 may also supply the coded data of the extracted displacement vectors to the displacement vector decoding unit 615. Furthermore, the demultiplexing unit 611 may supply the coded data of the extracted attributes to the attribute decoding unit 617 .

[0256] The atlas information decoding unit 612 performs processing related to decoding of the atlas information. For example, the atlas information decoding unit 612 may acquire coded data of the atlas information supplied from the demultiplexing unit 611. The atlas information decoding unit 612 may also decode the acquired coded data of the atlas information to generate (restore) the atlas information. This atlas information includes reconstruction control information, which includes target region designation information that uses texture information and control information. Therefore, the atlas information decoding unit 612 can also be referred to as a reconstruction control information decoding unit. Similarly, the atlas information decoding unit 612 can also be referred to as a target region designation information decoding unit. Similarly, the atlas information decoding unit 612 can also be referred to as a control information decoding unit. The atlas information decoding unit 612 may supply the generated atlas information (including the reconstruction control information) to the base mesh decoding unit 613. The atlas information decoding unit 612 may supply the generated atlas information (including the reconstruction control information) to the subdivision unit 614. The atlas information decoding unit 612 may supply the generated atlas information (including the reconstruction control information) to a displacement vector decoding unit 615. The atlas information decoding unit 612 may supply the generated atlas information (including the reconstruction control information) to a displacement vector application unit 616. The atlas information decoding unit 612 may supply the generated atlas information (including the reconstruction control information) to an attribute decoding unit 617. The atlas information decoding unit 612 may supply the generated atlas information (including the reconstruction control information) to an attribute application unit 618.

[0257] The base mesh decoding unit 613 performs processing related to decoding of the base mesh. For example, the base mesh decoding unit 613 may acquire coded data of the base mesh supplied from the demultiplexing unit 611. The base mesh decoding unit 613 may also decode the acquired coded data of the base mesh using a predetermined decoding method (e.g., Draco) to generate (restore) a base mesh (e.g., a vertex list, a triangle list, etc.). In this case, the base mesh decoding unit 613 may acquire atlas information (including reconstruction control information) supplied from the atlas information decoding unit 612 and decode the coded data of the base mesh based on the reconstruction control information. For example, the base mesh decoding unit 613 may decode a bitstream of the base mesh using an encoding method specified by the reconstruction control information. The base mesh decoding unit 613 may also supply the generated base mesh to the subdivision unit 614.

[0258] The subdivision unit 614 performs processing related to the subdivision of triangles of a base mesh. For example, the subdivision unit 614 may acquire a base mesh supplied from the base mesh decoding unit 613. The subdivision unit 614 may subdivide the base mesh (triangles thereof) to generate division points. In this case, the subdivision unit 614 may acquire atlas information (including reconstruction control information) supplied from the atlas information decoding unit 612 and subdivide the base mesh based on the reconstruction control information. For example, the subdivision unit 614 may subdivide the target region of the base mesh by the number of divisions specified by the reconstruction control information. The subdivision unit 614 may supply the subdivided base mesh to the displacement vector application unit 616.

[0259] The displacement vector decoding unit 615 performs processing related to the decoding of displacement vectors. For example, the displacement vector decoding unit 615 may acquire encoded data of displacement vectors (i.e., a displacement bitstream) supplied from the demultiplexing unit 611. The displacement vector decoding unit 615 may decode the encoded data of the displacement vectors to generate (restore) displacement vectors. For example, if the displacement vectors are encoded as displacement video, the displacement vector decoding unit 615 may decode the encoded data of the displacement vectors using a predetermined decoding method for 2D video, generate (restore) the displacement video, and unpack the displacement vectors from a displacement map, which is a frame image of the displacement video. Furthermore, if the displacement video is arithmetically coded, the displacement vector decoding unit 615 arithmetically decodes the encoded data of the displacement vectors to generate displacement vectors. In this case, the displacement vector decoding unit 615 may acquire atlas information (including reconstruction control information) supplied from the atlas information decoding unit 612 and decode the displacement vectors based on the reconstruction control information. The displacement vector decoding unit 615 may supply the displacement vector thus obtained to the displacement vector application unit 616 .

[0260] The displacement vector application unit 616 performs processing related to application of displacement vectors to the subdivided base mesh. For example, the displacement vector application unit 616 may obtain the subdivided base mesh supplied from the subdivision unit 614. The displacement vector application unit 616 may obtain displacement vectors supplied from the displacement vector decoding unit 615. The displacement vector application unit 616 may apply displacement vectors to the vertices of the subdivided base mesh. In other words, the displacement vector application unit 616 may generate a decoded mesh. In this case, the displacement vector application unit 616 may obtain atlas information (including reconstruction control information) supplied from the atlas information decoding unit 612 and apply displacement vectors to the vertices of the subdivided base mesh based on the reconstruction control information. The displacement vector application unit 616 may supply the decoded mesh generated in this manner to the attribute application unit 618.

[0261] The attribute decoding unit 617 performs processing related to attribute decoding. For example, the attribute decoding unit 617 may acquire coded attribute data supplied from the demultiplexing unit 611. The attribute decoding unit 617 may also decode the acquired coded attribute data using a predetermined decoding method for 2D video to generate (restore) attribute video. In this case, the attribute decoding unit 617 may acquire atlas information (including reconstruction control information) supplied from the atlas information decoding unit 612 and decode attributes based on the reconstruction control information. The attribute decoding unit 617 may also supply an attribute map, which is a frame image of the generated attribute video, to the attribute application unit 618.

[0262] The attribute application unit 618 performs processing related to the application of attributes to the decoded mesh. For example, the attribute application unit 618 may acquire the decoded mesh supplied from the displacement vector application unit 616. The attribute application unit 618 may acquire an attribute map supplied from the attribute decoding unit 617. The attribute application unit 618 may apply attributes of the attribute map to the decoded mesh. In this case, the attribute application unit 618 may acquire atlas information (including reconstruction control information) supplied from the atlas information decoding unit 612 and apply attributes to the decoded mesh based on the reconstruction control information. The attribute application unit 618 may supply the decoded mesh to which the attributes have been applied to the display processing unit 619.

[0263] The display processing unit 619 performs processing related to mesh display. For example, the display processing unit 619 may acquire a decoded mesh to which attributes have been applied, supplied from the attribute application unit 618. The display processing unit 619 may render the acquired decoded mesh and generate a display image for displaying the decoded mesh. The display processing unit 619 may then supply the generated display image to an external device outside the decoding device 600, and cause the display image to be displayed by another device or the like.

[0264] In the decoding device 600 (fourth information processing device), the atlas information decoding unit 612 applies method 2 to decode a bitstream of reconstruction control information that includes target region designation information that designates a target region for controlling processing related to mesh reconstruction and control information that indicates control over the target region. This target region designation information designates the target region using texture information related to the mesh texture. The base mesh decoding unit 613, the subdivision unit 614, the displacement vector decoding unit 615, and the displacement vector application unit 616 decode the bitstreams of the base mesh and displacement vectors in accordance with the reconstruction control information and reconstruct a decoded mesh.

[0265] With this configuration, the decoding device 600 can control each process included in the reconstruction process using reconstruction control information including target region designation information and control information that uses texture information. Therefore, even if the target region is made smaller to suppress a decrease in the performance of the reconstruction control, the increase in the amount of code can be suppressed compared to the case of sub-mesh units. In other words, the decoding device 600 can suppress a decrease in coding efficiency and a decrease in the performance of the reconstruction control.

[0266] <Flow of Decoding Process> An example of the flow of the decoding process executed by the decoding device 600 will be described with reference to the flowchart of FIG.

[0267] When the decoding process starts, the demultiplexing unit 611 of the decoding device 600 demultiplexes the V-DMC bitstream in step S601.

[0268] In step S602, the atlas information decoding unit 612 applies method 2 to decode the coded data of the atlas information and generate (restore) atlas information including reconstruction control information. That is, the atlas information decoding unit 612 decodes a bit stream of reconstruction control information including target region designation information that designates a target region for control of processing related to mesh reconstruction and control information that indicates control over that target region. This target region designation information is information that designates a target region using texture information related to the texture of the mesh.

[0269] In step S603, the base mesh decoding unit 613, the subdivision unit 614, the displacement vector decoding unit 615, the displacement vector application unit 616, the attribute decoding unit 617, and the attribute application unit 618 apply method 2 to identify the target area based on the target area designation information.

[0270] In step S604, the base mesh decoding unit 613, the subdivision unit 614, the displacement vector decoding unit 615, and the displacement vector application unit 616 apply method 2 to decode the base mesh and the displacement vector based on the reconstruction control information, and reconstruct the decoded mesh. That is, the base mesh decoding unit 613 decodes the encoded data of the base mesh based on the reconstruction control information to generate (reconstruct) the base mesh, the subdivision unit 614 subdivides the base mesh based on the reconstruction control information, the displacement vector decoding unit 615 decodes the displacement bitstream based on the reconstruction control information to generate (reconstruct) the displacement vector, and the displacement vector application unit 616 applies the displacement vector to the subdivided base mesh based on the reconstruction control information. In this way, the decoded mesh is reconstructed.

[0271] In step S605, the attribute decoding unit 617 decodes the attributes of all the regions. That is, the attribute decoding unit 417 decodes the coded data of the attributes of all the regions and generates (restores) the attributes of all the regions.

[0272] In step S606, the attribute application unit 618 applies the attributes of the entire region to the decoded mesh.

[0273] In step S607, the display processing unit 619 renders the decoded mesh to which the attributes have been applied to generate a display image.

[0274] When the process of step S607 is completed, the decoding process is completed. The decoding device 600 executes such a decoding process for each frame of the original mesh.

[0275] By performing each process in this manner, the decoding device 600 can control each process included in the reconstruction-related process using reconstruction control information including target region designation information and control information that utilizes texture information. Therefore, even if the target region is made smaller to suppress a decrease in the performance of reconstruction control, the increase in the amount of code can be suppressed compared to the case of sub-mesh units. In other words, the decoding device 600 can suppress a decrease in coding efficiency and a decrease in the performance of reconstruction control.

[0276] <10. Supplementary Notes> <Polygon Shape> In the above description, the polygon shape is described as being triangular, but this shape is merely an example. The polygon shape may be any polygonal shape.

[0277] <Encoding Method> In the above, V-DMC has been used as an example of an encoding method to which the present technology can be applied, but the present technology is not limited to this example, and can be applied to any encoding method that encodes a base mesh, a displacement vector, an attribute map including texture, atlas information, or information equivalent thereto.

[0278] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0279] FIG. 31 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0280] In a computer 900 shown in FIG. 31, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0281] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0282] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 912 includes, for example, a display, a speaker, and an output terminal. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0283] In a computer configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0284] The program executed by the computer can be applied by recording it on, for example, a removable medium 921 such as a package medium. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by inserting the removable medium 921 into the drive 915.

[0285] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0286] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0287] <Application of the Present Technology> The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0288] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set in which other functions are added to a unit (e.g., a video set).

[0289] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0290] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0291] <Fields and uses to which this technology can be applied> Systems, devices, processing units, etc. to which this technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, the uses thereof are also arbitrary.

[0292] <Others> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in a bitstream, but also in a bitstream that includes differential information of the identification information relative to certain reference information. Therefore, in this specification, "flag" and "identification information" encompass not only the information itself, but also differential information relative to the reference information.

[0293] Furthermore, various types of information (e.g., metadata) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, the associated pieces of data may be combined into one piece of data or may be separate pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Furthermore, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer not to the entire data, but to only part of the data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0294] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0295] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0296] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0297] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0298] Also, for example, each step of a single flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, multiple processes included in a single step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0299] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0300] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0301] The present technology can also be configured as follows. (1) An information processing device including: a priority control information generation unit that generates priority control information, which is information regarding the priority of processing related to mesh reconstruction; and an encoding unit that encodes a base mesh, a displacement vector, and the priority control information and generates a bitstream, wherein the base mesh is a mesh with lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, and the displacement vector is vector information indicating the displacement of vertices of the subdivided base mesh. (2) The information processing device described in (1), wherein the priority control information includes priority information indicating the priority of processing related to the reconstruction of the target region. (3) The information processing device described in (2), wherein the priority information includes a priority flag that indicates whether processing related to the reconstruction of the target region is to be prioritized. (4) The information processing device described in (2) or (3), wherein the priority information includes information indicating the order of priority of processing related to the reconstruction of the target region. (5) The information processing device according to any one of (1) to (4), wherein the priority control information includes target area designation information that designates a target area of ​​the priority control information. (6) The information processing device according to (5), wherein the target area designation information includes mesh range designation information that indicates the range of the target area in the mesh. (7) The information processing device according to (6), wherein the mesh range designation information includes a submesh ID that identifies a submesh. (8) The information processing device according to (6) or (7), wherein the mesh range designation information includes a face group ID that identifies a face group. (9) The information processing device according to any one of (6) to (8), wherein the mesh range designation information includes a list of vertex or face indices of the mesh. (10) The information processing device according to any one of (6) to (9), wherein the mesh range designation information includes a list of the number of faces that belong to the target area of ​​the mesh.(11) The information processing device according to any one of (6) to (10), wherein the mesh range designation information designates the range of the target region in the mesh by using texture information related to the texture of the mesh. (12) The information processing device according to (11), wherein the mesh range designation information includes a UV connected region ID that identifies a UV connected region corresponding to the target region in a two-dimensional plane in which the texture is packed. (13) The information processing device according to (11), wherein the mesh range designation information includes a patch ID that identifies a patch corresponding to the target region in a two-dimensional plane in which the texture is packed using an orthoAtlas. (14) The information processing device according to any one of (5) to (13), wherein the target region designation information includes displacement range designation information that indicates a range of the displacement vector corresponding to the target region. (15) The information processing device according to (14), wherein the displacement range designation information includes a displacement bitstream ID that identifies a bitstream of the displacement vector corresponding to the target region. (16) The information processing device according to (14) or (15), wherein the displacement range designation information includes a list of start pixel indices indicating start positions of the displacement vectors corresponding to the target region in a two-dimensional plane in which the displacement vectors are packed, or a list of in-range pixel counts indicating the number of pixels corresponding to the target region. (17) The information processing device according to any of (14) to (16), wherein the displacement range designation information includes a list of start block indices indicating start positions of the displacement vectors corresponding to the target region in a two-dimensional plane in which the displacement vectors are packed, or a list of in-range block counts indicating the number of blocks corresponding to the target region. (18) The information processing device according to any of (1) to (17), wherein the encoding unit generates the bitstream including a priority bitstream obtained by encoding the displacement vectors of a priority region, and the priority region is a region in which processing related to the reconstruction is prioritized over non-priority regions.(19) The information processing device according to any of (1) to (17), wherein the encoding unit generates the bitstream including a priority bitstream in which the displacement vectors of a priority region are encoded and a non-priority bitstream in which the displacement vectors of a non-priority region are encoded, the priority region being an area in which processing related to the reconstruction is prioritized over the non-priority region. (20) The information processing device according to any of (1) to (17), wherein the encoding unit generates the bitstream including a priority bitstream in which the displacement vectors of a priority region are encoded and a full-area bitstream in which the displacement vectors of all regions are encoded, the priority region being an area in which processing related to the reconstruction is prioritized over the non-priority region. (21) The information processing device according to any of (1) to (17), wherein the encoding unit generates the bitstream including a full-area bitstream in which the displacement vectors of all regions are encoded, in which the displacement vectors of the priority region and the displacement vectors of the non-priority region are packed independently of each other, the priority region being an area in which processing related to the reconstruction is prioritized over the non-priority region. (22) The information processing device according to any one of (1) to (21), wherein the priority control information includes information regarding a priority for subdivision of the base mesh. (23) The information processing device according to any one of (1) to (22), wherein the priority control information includes information regarding a priority for decoding the displacement vector. (24) The information processing device according to any one of (1) to (23), wherein the priority control information includes information regarding a priority for application of the displacement vector. (25) An information processing method comprising: generating priority control information that is information regarding a priority for processing related to mesh reconstruction; and encoding a base mesh, the displacement vector, and the priority control information to generate a bitstream, wherein the base mesh is a mesh with lower resolution than an original mesh to be coded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and wherein the displacement vector is vector information indicating the displacement of vertices of the subdivided base mesh.

[0302] (41) An information processing device comprising: a priority control information decoding unit that decodes a bit stream of priority control information that is information regarding the priority of processing related to mesh reconstruction; a subdivision unit that subdivides a base mesh as processing related to the reconstruction; and a displacement vector application unit that applies a displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information as processing related to the reconstruction, wherein the base mesh is a mesh with lower resolution than an original mesh to be coded that is composed of vertices and connections that represent a three-dimensional structure of an object and is generated by thinning out vertices from the original mesh, and the displacement vector is vector information that indicates displacement of vertices of the subdivided base mesh. (42) The information processing device according to (41), wherein the priority control information includes priority information that indicates a priority of processing related to the reconstruction of a target region, and the displacement vector application unit applies the displacement vector to the target region of the base mesh in accordance with the priority information. (43) The information processing device according to (42), wherein the priority information includes a priority flag indicating whether or not to prioritize processing related to the reconstruction of the target region, and the displacement vector application unit applies the displacement vector to the target region of the base mesh for which the priority flag is true, in preference to the target region for which the priority flag is false. (44) The information processing device according to (42) or (43), wherein the priority information includes information indicating an order of priority for processing related to the reconstruction of the target region, and the displacement vector application unit applies the displacement vector to the target region of the base mesh in accordance with the order of priority. (45) The information processing device according to any of (41) to (44), wherein the priority control information includes object region designation information that designates a target region of the priority control information. (46) The information processing device according to (45), wherein the object region designation information includes mesh range designation information that indicates the range of the object region in the mesh. (47) The information processing device according to (46), wherein the mesh range designation information includes a submesh ID that identifies a submesh.(48) The information processing device according to (46) or (47), wherein the mesh range designation information includes a face group ID for identifying a face group. (49) The information processing device according to any of (46) to (48), wherein the mesh range designation information includes a list of indices of vertices or faces of the mesh. (50) The information processing device according to any of (46) to (49), wherein the mesh range designation information includes a list of the number of faces belonging to the target area of ​​the mesh. (51) The information processing device according to any of (46) to (50), wherein the mesh range designation information specifies the range of the target area in the mesh by using texture information related to a texture of the mesh. (52) The information processing device according to (51), wherein the mesh range designation information includes a UV-connected area ID for identifying a UV-connected area corresponding to the target area in a two-dimensional plane in which the texture is packed. (53) The information processing device according to (51), wherein the mesh range designation information includes a patch ID for identifying a patch corresponding to the target area in a two-dimensional plane in which the texture is packed using an orthoAtlas. (54) The information processing device according to any of (45) to (53), wherein the target area designation information includes displacement range designation information indicating a range of the displacement vector corresponding to the target area. (55) The information processing device according to (54), wherein the displacement range designation information includes a displacement bitstream ID that identifies a bitstream of the displacement vector corresponding to the target area. (56) The information processing device according to (54) or (55), wherein the displacement range designation information includes a list of start pixel indexes that indicate start positions of the displacement vector corresponding to the target area in a two-dimensional plane in which the displacement vectors are packed, or a list of within-range pixel numbers that indicate the number of pixels corresponding to the target area.(57) The information processing device according to any of (54) to (56), wherein the displacement range designation information includes a list of start block indexes indicating start positions of the displacement vectors corresponding to the target region in a two-dimensional plane in which the displacement vectors are packed, or a list of in-range block numbers indicating the number of blocks corresponding to the target region. (58) The information processing device according to any of (41) to (57), wherein the information processing device further includes a displacement vector decoding unit that, as the reconstruction-related processing, decodes a priority bit stream that is a bit stream of the displacement vectors of a priority region that is prioritized over a non-priority region, and generates the displacement vectors, and the displacement vector application unit is configured to apply the generated displacement vectors to the subdivided base mesh in accordance with a priority order based on the priority control information. (59) An information processing device according to any of (41) to (57), further comprising a displacement vector decoding unit that decodes a displacement bit stream including a priority bit stream and a non-priority bit stream as part of the reconstruction-related processing, and generates the displacement vector; wherein the displacement vector application unit is configured to apply the generated displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information; the displacement bit stream is a bit stream of the displacement vector; the priority bit stream is the displacement bit stream of a priority region in which the reconstruction-related processing is prioritized over a non-priority region; and the non-priority bit stream is the displacement bit stream of the non-priority region.(60) An information processing device according to any of (41) to (57), further comprising a displacement vector decoding unit that decodes a displacement bit stream including a priority bit stream and a full-area bit stream as part of the reconstruction-related processing, and generates the displacement vector; wherein the displacement vector application unit is configured to apply the generated displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information; the displacement bit stream is a bit stream of the displacement vector; the priority bit stream is the displacement bit stream of a priority area in which the reconstruction-related processing is prioritized over non-priority areas; and the full-area bit stream is the displacement bit stream of the full area. (61) The information processing device according to any one of (41) to (57), further comprising a displacement vector decoding unit configured to decode an entire region bit stream in which the displacement vectors of a priority region and the displacement vectors of a non-priority region are packed independently of each other as a bit stream of the displacement vectors of the entire region as a process related to the reconstruction, and generate the displacement vectors, wherein the displacement vector application unit is configured to apply the generated displacement vectors to the subdivided base mesh in accordance with a priority order based on the priority control information, and the priority region is a region in which the process related to the reconstruction is prioritized over the non-priority region. (62) The information processing device according to any one of (41) to (61), wherein the priority control information includes information related to a priority of subdivision of the base mesh, and the subdivision unit subdivides the priority region of the base mesh specified by the priority control information with priority over the non-priority region of the base mesh.(63) The information processing device according to any of (41) to (62), wherein the priority control information includes information regarding priority of decoding of the displacement vector, and further comprises a displacement vector decoding unit that decodes a displacement bit stream that is a bit stream of the displacement vector as a process related to the reconstruction, and generates the displacement vector of a priority region specified by the priority control information with priority over the displacement vector of a non-priority region, and the displacement vector application unit is configured to apply the generated displacement vector to the subdivided base mesh. (64) The information processing device according to any of (41) to (63), wherein the priority control information includes information regarding priority of application of the displacement vector, and the displacement vector application unit is configured to apply the displacement vector of the priority region specified by the priority control information to the subdivided base mesh with priority over the displacement vector of a non-priority region. (65) An information processing method comprising: decoding a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction; subdividing a base mesh as the processing related to the reconstruction; and applying a displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information as the processing related to the reconstruction, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, and the displacement vector is vector information indicating the displacement of the vertices of the subdivided base mesh.

[0303] (81) An information processing device comprising: a target region designation information generation unit that generates target region designation information that designates a target region for control of processing related to mesh reconstruction, a control information generation unit that generates control information that indicates control of processing related to the reconstruction for the target region, and an encoding unit that encodes a base mesh, a displacement vector, and reconstruction control information that includes the target region designation information and the control information, and generates a bitstream, wherein the base mesh is a mesh with lower resolution than an original mesh to be coded that is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, the displacement vector is vector information that indicates displacement of vertices of the subdivided base mesh, and the target region designation information designates the target region using texture information related to a texture of the mesh. (82) The information processing device according to (81), wherein the target region designation information includes a UV-connected region ID that identifies a UV-connected region that corresponds to the target region in a two-dimensional plane in which the texture is packed. (83) The information processing device according to (81), wherein the target region designation information includes a patch ID that identifies a patch corresponding to the target region in a two-dimensional plane in which the texture is packed by orthoAtlas. (84) The information processing device according to any of (81) to (83), wherein the reconstruction control information includes control information for controlling subdivision of the target region of the base mesh. (85) The information processing device according to any of (81) to (84), wherein the reconstruction control information includes control information for controlling decoding of a bit stream of the displacement vector of the target region. (86) The information processing device according to any of (81) to (85), wherein the reconstruction control information includes control information for controlling application of the displacement vector to the target region of the subdivided base mesh. (87) The information processing device according to any of (81) to (86), wherein the reconstruction control information includes control information for controlling decoding of a bit stream of the target region of the base mesh.(88) An information processing method comprising: generating target area designation information that designates a target area for control of processing related to mesh reconstruction; generating control information that indicates control of processing related to the reconstruction for the target area; encoding a base mesh, a displacement vector, and reconstruction control information including the target area designation information and the control information, and generating a bitstream, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; the displacement vector is vector information that indicates the displacement of vertices of the subdivided base mesh; and the target area designation information designates the target area by utilizing texture information related to the texture of the mesh.

[0304] (101) A reconstruction control information decoding unit that decodes a bit stream of reconstruction control information including target area designation information that designates a target area for control of processing related to mesh reconstruction and control information that indicates control of processing related to the reconstruction for the target area; a base mesh decoding unit that executes a base mesh decoding process that decodes a bit stream of a base mesh as the processing related to the reconstruction; a subdivision unit that executes a subdivision process that subdivides the base mesh obtained by decoding as the processing related to the reconstruction; a displacement vector decoding unit that executes a displacement vector decoding process that decodes a bit stream of displacement vectors as the processing related to the reconstruction; and a displacement vector application unit that executes a displacement vector application process that applies the displacement vector obtained by decoding to the subdivided base mesh as the processing related to the reconstruction, wherein the base mesh is a mesh with lower resolution than the original mesh that is to be coded and is composed of vertices and connections that represent a three-dimensional structure of an object, and the displacement vector is vector information that indicates displacement of vertices of the subdivided base mesh; the target area designation information designates the target area using texture information related to the texture of the mesh; The information processing device according to claim 1, wherein at least one of the base mesh decoding process, the subdivision process, the displacement vector decoding process, and the displacement vector application process is executed in accordance with the reconstruction control information. (102) The information processing device according to claim 1, wherein the target region designation information includes a UV connected region ID that identifies a UV connected region corresponding to the target region in a two-dimensional plane in which the texture is packed. (103) The information processing device according to claim 1, wherein the target region designation information includes a patch ID that identifies a patch corresponding to the target region in a two-dimensional plane in which the texture is packed using an orthoAtlas.(104) The information processing device according to any of (101) to (103), wherein the reconstruction control information includes the control information for controlling the subdivision process for the target region of the base mesh, and the subdivision unit executes the subdivision process for the target region of the base mesh in accordance with the control information of the reconstruction control information. (105) The information processing device according to any of (101) to (104), wherein the reconstruction control information includes the control information for controlling the displacement vector decoding process for the displacement vector of the target region, and the displacement vector decoding unit executes the displacement vector decoding process for the displacement vector of the target region in accordance with the control information of the reconstruction control information. (106) The information processing device according to any of (101) to (105), wherein the reconstruction control information includes the control information for controlling the displacement vector application process for the displacement vector of the target region, and the displacement vector application unit executes the displacement vector application process for the displacement vector of the target region in accordance with the control information of the reconstruction control information. (107) The information processing device according to any one of (101) to (106), wherein the reconstruction control information includes the control information for controlling the base mesh decoding process for the target region of the base mesh, and the base mesh decoding unit executes the base mesh decoding process for the target region of the base mesh according to the control information of the reconstruction control information.(108) A method for encoding a mesh comprising: decoding a bit stream of reconstruction control information including target area designation information designating a target area for controlling a process related to reconstruction of a mesh and control information indicating control of the process related to reconstruction for the target area; executing, as the process related to reconstruction, a base mesh decoding process for decoding a bit stream of a base mesh; executing, as the process related to reconstruction, a subdivision process for subdividing the base mesh obtained by decoding; executing, as the process related to reconstruction, a displacement vector decoding process for decoding a bit stream of displacement vectors; and executing, as the process related to reconstruction, a displacement vector application process for applying the displacement vector obtained by decoding to the subdivided base mesh, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of the object, and is generated by thinning out vertices from the original mesh; the displacement vector is vector information indicating the displacement of vertices of the subdivided base mesh; the target area designation information designates the target area using texture information related to the texture of the mesh; At least one of the base mesh decoding process, the subdivision process, the displacement vector decoding process, and the displacement vector application process is performed in accordance with the reconstruction control information.

[0305] 300 Encoding device, 311 Preprocessing unit, 312 V-DMC encoding unit, 321 Base mesh generation unit, 322 Atlas information generation unit, 323 Priority control information generation unit, 324 Displacement vector generation unit, 351 Atlas information encoding unit, 352 Base mesh encoding unit, 353 Displacement vector correction unit, 354 Displacement vector encoding unit, 355 Mesh reconstruction unit, 356 Attribute map conversion unit, 357 Attribute encoding unit, 358 Multiplexing unit, 400 Decoding device, 411 Demultiplexing unit, 412 Atlas information decoding unit, 413 Base mesh decoding unit, 414 Subdivision unit, 415 Displacement vector decoding unit, 416 Displacement vector application unit, 417 Attribute decoding unit, 418 Attribute application unit, 419 Display processing unit, 500 encoding device, 511 preprocessing unit, 512 V-DMC encoding unit, 521 base mesh generation unit, 522 atlas information generation unit, 523 reconstruction control information generation unit, 524 displacement vector generation unit, 531 target region designation information generation unit, 532 control information generation unit, 551 atlas information encoding unit, 552 base mesh encoding unit, 553 displacement vector correction unit, 554 displacement vector encoding unit, 555 mesh reconstruction unit, 556 attribute map conversion unit, 557 attribute encoding unit, 558 multiplexing unit, 600 decoding device, 611 demultiplexing unit, 612 atlas information decoding unit, 613 base mesh decoding unit, 614 subdivision unit, 615 displacement vector decoding unit, 616 displacement vector application unit 617 Attribute decoding unit, 618 Attribute application unit, 619 Display processing unit, 900 Computer

Claims

1. An information processing device comprising: a priority control information decoding unit that decodes a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction; a subdivision unit that subdivides a base mesh as part of the processing related to the reconstruction; and a displacement vector application unit that applies a displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information as part of the processing related to the reconstruction, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, and the displacement vector is vector information that indicates the displacement of the vertices of the subdivided base mesh.

2. The information processing device according to claim 1, wherein the priority control information includes priority information indicating the priority of processing related to the reconstruction of the target region, and the displacement vector application unit applies the displacement vector to the target region of the base mesh in accordance with the priority information.

3. The information processing device described in claim 2, wherein the priority information includes a priority flag indicating whether or not processing related to the reconstruction of the target area is to be prioritized, and the displacement vector application unit applies the displacement vector to the target area for which the priority flag of the base mesh is true in priority over the target area for which the priority flag is false.

4. The information processing device described in claim 2, wherein the priority information includes information indicating the priority order of processing related to the reconstruction of the target area, and the displacement vector application unit applies the displacement vector to the target area of ​​the base mesh in accordance with the priority order.

5. The information processing device according to claim 1, wherein the priority control information includes target area designation information that designates a target area of ​​the priority control information.

6. The information processing device according to claim 5, wherein the target area designation information includes mesh range designation information indicating the range of the target area in the mesh.

7. The information processing device according to claim 6, wherein the mesh range designation information includes a submesh ID for identifying a submesh.

8. The information processing device according to claim 6, wherein the mesh range designation information designates the range of the target region in the mesh by utilizing texture information relating to the texture of the mesh.

9. The information processing device according to claim 8, wherein the mesh range designation information includes a UV connected area ID that identifies a UV connected area corresponding to the target area in the two-dimensional plane on which the texture is packed.

10. The information processing device according to claim 8, wherein the mesh range specification information includes a patch ID that identifies a patch corresponding to the target area in a two-dimensional plane in which the texture is packed by orthoAtlas.

11. The information processing device according to claim 5, wherein the target area designation information includes displacement range designation information indicating a range of the displacement vector corresponding to the target area.

12. An information processing device as described in claim 1, further comprising a displacement vector decoding unit that decodes a displacement bit stream including a priority bit stream and a non-priority bit stream as part of the reconstruction-related processing, and generates the displacement vector; the displacement vector application unit is configured to apply the generated displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information; the displacement bit stream is a bit stream of the displacement vector; the priority bit stream is the displacement bit stream of a priority region in which the reconstruction-related processing is prioritized over a non-priority region; and the non-priority bit stream is the displacement bit stream of the non-priority region.

13. An information processing device as described in claim 1, further comprising a displacement vector decoding unit that decodes a displacement bit stream including a priority bit stream and a full-area bit stream as part of the reconstruction-related processing, and generates the displacement vector; the displacement vector application unit is configured to apply the generated displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information; the displacement bit stream is a bit stream of the displacement vector; the priority bit stream is the displacement bit stream of a priority area in which the reconstruction-related processing is prioritized over non-priority areas; and the full-area bit stream is the displacement bit stream of the full area.

14. The information processing device of claim 1, further comprising a displacement vector decoding unit that decodes a bit stream of the displacement vectors of the entire region, in which the displacement vectors of the priority region and the displacement vectors of the non-priority region are packed independently of each other, as a process related to the reconstruction, and generates the displacement vectors; the displacement vector application unit is configured to apply the generated displacement vectors to the subdivided base mesh in accordance with a priority order based on the priority control information; and the priority region is a region in which the process related to the reconstruction is prioritized over the non-priority region.

15. An information processing device as described in claim 1, wherein the priority control information includes information regarding the priority of subdivision of the base mesh, and the subdivision unit subdivides the priority area of ​​the base mesh specified by the priority control information in priority over non-priority areas of the base mesh.

16. The information processing device of claim 1, wherein the priority control information includes information regarding the priority of decoding the displacement vector, and further comprises a displacement vector decoding unit that, as a process related to the reconstruction, decodes a displacement bit stream that is a bit stream of the displacement vector and generates the displacement vector of a priority area specified by the priority control information by giving priority to the displacement vector of a non-priority area, and the displacement vector application unit is configured to apply the generated displacement vector to the subdivided base mesh.

17. The information processing device according to claim 1, wherein the priority control information includes information regarding the priority of application of the displacement vector, and the displacement vector application unit is configured to apply the displacement vector of a priority region specified by the priority control information to the subdivided base mesh in priority over the displacement vector of a non-priority region.

18. An information processing method comprising: decoding a bit stream of priority control information, which is information regarding the priority of processing related to mesh reconstruction; subdividing a base mesh as processing related to the reconstruction; and applying a displacement vector to the subdivided base mesh in accordance with a priority order based on the priority control information as processing related to the reconstruction, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, and the displacement vector is vector information indicating the displacement of the vertices of the subdivided base mesh.

19. An information processing device comprising: a priority control information generation unit that generates priority control information, which is information regarding the priority of processing related to mesh reconstruction; and an encoding unit that encodes a base mesh, a displacement vector, and the priority control information and generates a bit stream, wherein the base mesh is a mesh with lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh, and the displacement vector is vector information that indicates the displacement of the vertices of the subdivided base mesh.

20. An information processing method comprising: generating priority control information, which is information regarding the priority of processing related to mesh reconstruction; encoding a base mesh, a displacement vector, and the priority control information to generate a bitstream; wherein the base mesh is a mesh with lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out vertices from the original mesh; and the displacement vector is vector information indicating the displacement of vertices of the subdivided base mesh.

Images