Information processing device and method

Abstract

The present disclosure relates to an information processing device and method which make it possible to suppress a reduction in coding efficiency. The method includes: decoding encoded data; generating motion vectors indicating positional differences between the vertices of a reference base mesh and the vertices of a current base mesh; reconstructing the current base mesh by applying the motion vectors to the vertices of the reference base mesh; and reconstructing a current mesh by applying a reference displacement vector to the vertices of the current base mesh. In addition, the motion vectors indicating the positional differences between the vertices of the reference base mesh and the vertices of the current base mesh are encoded; and control information relating to the application of the reference displacement vector to the vertices of the current base mesh in the reconfiguration of the current mesh is encoded. The present disclosure is applicable, for example, to information processing devices, electronic instruments, information processing methods and programs.

Description

Information processing device and method

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that can suppress a decrease in coding efficiency.

[0002] Conventionally, V-DMC (Video-based Dynamic Mesh Coding) has been used as a method for encoding meshes, which are 3D data that represent the three-dimensional structure of an object using vertices and connections (see, for example, Non-Patent Document 1). In V-DMC, the mesh to be encoded is represented by a coarse base mesh and displacement vectors at division points obtained by subdividing the base mesh, and the base mesh and displacement vectors are encoded. The displacement vectors are stored (packed) in two-dimensional images, and the two-dimensional images are used as frames to encode a moving image (displacement video).

[0003] Various other coding methods that apply V-DMC have been proposed, such as a method of arithmetically coding displacement coefficients or their prediction residuals (see, for example, Non-Patent Document 2), and a method of skipping transmission of the current motion vector (see, for example, Non-Patent Document 3).

[0004] Khaled Mammou, Jungsun Kim, Alexis Tourapis, Dimitri Podborski, Krasimir Kolarov, "[V-CG] Apple's Dynamic Mesh Coding CfP Response", ISO / IEC JTC 1 / SC 29 / WG 7 m59281, April 2022Chao Huang, Xiang Zhang, Xiaozhong Xu, Jun Tian, ​​Shan Liu, "Arithmetic Coding of Displacements Jianfeng Xu, Kei Kawamura, "[V-DMC][new] On supporting a SKIP type in base mesh coding", ISO / IEC JTC 1 / SC 29 / WG 7 m62092, January 2023

[0005] However, applying these methods does not always result in the best coding efficiency, and other methods have been sought.

[0006] The present disclosure has been made in light of such circumstances, and makes it possible to suppress a decrease in coding efficiency.

[0007] An information processing device according to one aspect of the present technology includes: a motion vector decoding unit that decodes encoded data and generates motion vectors that indicate positional differences between vertices of a reference base mesh and vertices of a current base mesh; a current base mesh reconstruction unit that reconstructs the current base mesh by applying the motion vectors to the vertices of the reference base mesh; and a current mesh reconstruction unit that reconstructs the current mesh by applying a reference displacement vector to the vertices of the current base mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame, and the current base mesh is a base mesh of the current frame, and the base mesh is a mesh of lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh, and the reference displacement vector indicates a positional difference between the vertices obtained by subdividing the reference base mesh and the vertices of the reference mesh.

[0008] An information processing method according to one aspect of the present technology includes: decoding encoded data; generating motion vectors indicating differences in position between vertices of a reference base mesh and vertices of a current base mesh; reconstructing the current base mesh by applying the motion vectors to the vertices of the reference base mesh; and reconstructing a current mesh by applying a reference displacement vector to the vertices of the current base mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame; the current base mesh is a base mesh of the current frame; the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh; and the reference displacement vector indicates differences in position between vertices obtained by subdividing the reference base mesh and the vertices of the reference mesh.

[0009] According to another aspect of the present technology, there is provided an information processing device including: an encoding unit that encodes motion vectors indicating positional differences between vertices of a reference base mesh and vertices of a current base mesh; and a control information encoding unit that encodes control information related to application of reference displacement vectors to vertices of the current base mesh in reconstructing the current mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame; the current base mesh is a base mesh of the current frame; the base mesh is a mesh of lower resolution than an original mesh to be encoded, which is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh; and the reference displacement vector indicates a positional difference between vertices obtained by subdividing the reference base mesh and vertices of the reference mesh.

[0010] Another aspect of the present technology is an information processing method for encoding motion vectors indicating positional differences between vertices of a reference base mesh and vertices of a current base mesh, and encoding control information related to application of reference displacement vectors to vertices of the current base mesh in reconstructing the current mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame, the current base mesh is a base mesh of the current frame, the base mesh is a mesh of lower resolution than an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh, and the reference displacement vector indicates a positional difference between vertices obtained by subdividing the reference base mesh and the vertices of the reference mesh.

[0011] According to yet another aspect of the present technology, there is provided an information processing device comprising: a current displacement vector decoding unit that decodes encoded data and generates a current displacement vector that indicates a difference in position between the vertices of a current mesh and the vertices of a reference mesh; and a displacement vector application unit that reconstructs the current mesh by applying the current displacement vector to the vertices of the reference mesh, wherein the current mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh composed of vertices and connections that represent the three-dimensional structure of the object in a reference frame referenced from the current frame.

[0012] An information processing method according to yet another aspect of the present technology includes decoding encoded data, generating current displacement vectors indicating positional differences between vertices of a current mesh and vertices of a reference mesh, and reconstructing the current mesh by applying the current displacement vectors to the vertices of the reference mesh, wherein the current mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of the object in a reference frame referenced from the current frame.

[0013] An information processing device according to yet another aspect of the present technology includes a current displacement vector derivation unit that derives a current displacement vector indicating a difference in position between a vertex of a current mesh and a vertex of a reference mesh, and a current displacement vector encoding unit that encodes the current displacement vector, wherein the current mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of the object in a reference frame referenced from the current frame.

[0014] An information processing method according to yet another aspect of the present technology includes deriving a current displacement vector indicating a difference in position between a vertex of a current mesh and a vertex of a reference mesh, and encoding the current displacement vector, wherein the current mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of the object in a reference frame referenced from the current frame.

[0015] According to yet another aspect of the present technology, an information processing device includes an upconversion unit that upconverts a current base mesh, which is a base mesh of a current frame; a current displacement vector decoding unit that decodes encoded data and generates a current displacement vector that indicates a difference in position between the vertices of the current mesh and the vertices of the upconverted current base mesh; and a current mesh reconstruction unit that reconstructs the current mesh by applying the current displacement vector to the vertices of the upconverted current base mesh, wherein the base mesh is an information processing device that is generated by thinning out vertices from an original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and has a lower resolution than the original mesh.

[0016] An information processing method according to yet another aspect of the present technology includes up-converting a current base mesh that is a base mesh of a current frame, decoding encoded data, generating current displacement vectors that indicate differences in position between vertices of the current mesh and vertices of the up-converted current base mesh, and reconstructing the current mesh by applying the current displacement vectors to the vertices of the up-converted current base mesh, wherein the base mesh is a mesh of lower resolution than an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh.

[0017] According to yet another aspect of the present technology, an information processing device includes an upconversion unit that upconverts a current base mesh, which is a base mesh of a current frame; a current displacement vector derivation unit that derives a current displacement vector indicating a difference in position between a vertex of the current mesh and a vertex of the upconverted current base mesh; and a current displacement vector encoding unit that encodes the current displacement vector, wherein the base mesh is an information processing device that is a mesh of lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh to be encoded.

[0018] An information processing method according to yet another aspect of the present technology includes up-converting a current base mesh that is a base mesh of a current frame, deriving a current displacement vector that indicates a difference in position between the vertices of the current mesh and the vertices of the up-converted current base mesh, and encoding the current displacement vector, wherein the base mesh is a mesh of lower resolution than the original mesh that is generated by thinning out the vertices from an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object.

[0019] In an information processing device and method according to one aspect of the present technology, encoded data is decoded, motion vectors indicating the positional differences between the vertices of a reference base mesh and the vertices of a current base mesh are generated, the current base mesh is reconstructed by applying the motion vectors to the vertices of the reference base mesh, and the current mesh is reconstructed by applying reference displacement vectors to the vertices of the current base mesh.

[0020] In another aspect of the information processing device and method of the present technology, motion vectors indicating the positional differences between the vertices of a reference base mesh and the vertices of a current base mesh are encoded, and control information regarding the application of the reference displacement vectors to the vertices of the current base mesh in the reconstruction of the current mesh is encoded.

[0021] In an information processing device and method according to yet another aspect of the present technology, encoded data is decoded, a current displacement vector indicating the difference in position between the vertices of the current mesh and the vertices of the reference mesh is generated, and the current mesh is reconstructed by applying the current displacement vector to the vertices of the reference mesh.

[0022] In an information processing device and method according to still another aspect of the present technology, a current displacement vector indicating a difference in position between a vertex of a current mesh and a vertex of a reference mesh is derived, and the current displacement vector is encoded.

[0023] In an information processing device and method according to yet another aspect of the present technology, a current base mesh, which is a base mesh of a current frame, is upconverted, the encoded data is decoded to generate a current displacement vector indicating the difference in position between the vertices of the current mesh and the vertices of the upconverted current base mesh, and the current mesh is reconstructed by applying the current displacement vector to the vertices of the upconverted current base mesh.

[0024] In an information processing device and method according to yet another aspect of the present technology, a current base mesh, which is a base mesh of a current frame, is upconverted, a current displacement vector indicating the difference in position between the vertices of the current mesh and the vertices of the upconverted current base mesh is derived, and the current displacement vector is encoded.

[0025] 1 is a diagram for explaining meshes; 2 is a diagram for explaining V-DMC; 3 is a diagram for explaining an example of reconstruction; 4 is a diagram for explaining an example of a reconstruction; 5 is a diagram for explaining an example of a mesh encoding / decoding method; 6 is a diagram for explaining an example of a reconstruction; 7 is a diagram for explaining a key frame; 8 is a block diagram showing an example of the main configuration of an encoding device; 9 is a flowchart for explaining an example of the flow of inter encoding processing; 10 is a block diagram showing an example of the main configuration of a decoding device; 11 is a flowchart for explaining an example of the flow of inter decoding processing; 12 is a diagram for explaining an example of a reconstruction; 13 is a block diagram showing an example of the main configuration of an encoding device; 14 is a flowchart for explaining an example of the flow of inter encoding processing; 15 is a block diagram showing an example of the main configuration of a decoding device; 16 is a flowchart for explaining an example of the flow of inter decoding processing; 17 is a diagram for explaining an example of a reconstruction; 18 is a block diagram showing an example of the main configuration of an encoding device; 19 is a flowchart for explaining an example of the flow of inter encoding processing; 20 is a block diagram showing an example of the main configuration of a decoding device; 21 is a flowchart for explaining an example of the flow of inter decoding processing; 22 is a diagram for explaining an example of a reconstruction; 23 is a block diagram showing an example of the main configuration of an encoding device; 24 is a flowchart for explaining an example of the flow of inter encoding processing; 25 is a block diagram showing an example of the main configuration of a decoding device; 26 is a flowchart for explaining an example of the flow of inter decoding processing; Fig. 1 is a block diagram showing an example of the main configuration of a decoding device; Fig. 2 is a flowchart explaining an example of the flow of inter-decoding processing; Fig. 3 is a block diagram showing an example of the main configuration of a coding device; Fig. 4 is a flowchart explaining an example of the flow of encoding processing; and Fig. 5 is a block diagram showing an example of the main configuration of a computer.

[0026] The following describes modes for carrying out the present disclosure (hereinafter referred to as embodiments). The description will be made in the following order: 1. Literature supporting technical content and technical terminology 2. Encoding of current displacement vector 3. Reduction of current displacement vector transmission using reference displacement vector 4. Reduction of displacement vector information amount using up-conversion 5. Mode selection 6. Supplementary notes

[0027] <1. Literature, etc. supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the content described in the embodiments, but also the content described in the following non-patent documents, etc. that were publicly known at the time of filing, and the content of other documents referenced in the following non-patent documents.

[0028] Non-patent document 1: (described above) Non-patent document 2: (described above) Non-patent document 3: (described above)

[0029] In other words, the contents of the above-mentioned non-patent documents and the contents of other documents referenced in the above-mentioned non-patent documents are also used as the basis for determining the support requirements.

[0030] <2. Encoding of current displacement vector> <V-DMC> Conventionally, 3D data representing the three-dimensional structure of a three-dimensional structure (object with a three-dimensional shape) has been available as a mesh, which represents the three-dimensional shape of the object surface by forming polygons with vertices and connections.

[0031] As shown in the upper left of Figure 1, in a mesh, vertices 11 and connections 12 connecting these vertices 11 form polygonal planes (polygons). These polygons (also called faces) represent the surface of a three-dimensional object, i.e., the three-dimensional shape of the object. A texture 13 can be applied to each face of this mesh.

[0032] Mesh data is composed of information such as that shown in the lower part of Figure 1. Vertex information 14, shown first from the left in the lower part of Figure 1, is information indicating the three-dimensional position (three-dimensional coordinates (X, Y, Z)) of each vertex 11 that constitutes the mesh. Connection information 15, shown second from the left in the lower part of Figure 1, is information indicating each connection 12 that constitutes the mesh. A texture image 16, shown third from the left in the lower part of Figure 1, is map information for the texture 13 that is applied to each face. A UV map 17, shown fourth from the left in the lower part of Figure 1, is information indicating the correspondence between the vertices 11 and the texture 13. The UV map 17 indicates the coordinates (UV coordinates) of each vertex 11 in the texture image 16.

[0033] As an example of a method for encoding such meshes, there is V-DMC (Video-based Dynamic Mesh Coding) as disclosed in Non-Patent Document 1.

[0034] In V-DMC, a mesh to be coded is represented by a coarse base mesh and displacement vectors at division points obtained by subdividing the base mesh, and the base mesh and displacement vectors are coded.

[0035] For example, assume that there is an original mesh as shown in the top row of Figure 2. In this specification, the original mesh refers to the mesh to be encoded. In Figure 2, black dots represent vertices, and lines connecting the black dots represent connections. As mentioned above, a mesh essentially forms polygons by vertices and connections, but for convenience of explanation, it is described here as a group of vertices connected linearly (in series).

[0036] By decimating some of the vertices of the original mesh, the number of polygons is reduced, resulting in a coarse mesh as shown in the second row from the top of Figure 2. In this specification, such a coarse mesh obtained by decimating some of the vertex connections of the original mesh is referred to as a base mesh. In other words, the base mesh can be said to be a mesh with a reduced number of polygons from the original mesh.

[0037] By subdividing this base mesh, a subdivided base mesh is obtained, as shown in the third row from the top of Figure 2. In this specification, subdivision refers to the process of dividing the polygons of a mesh into multiple pieces. In other words, by subdividing a base mesh, the number of polygons in the base mesh increases. In other words, by subdividing a base mesh, vertices and connections are added to the base mesh. For example, by subdividing the base mesh so as to add the number of vertices that were thinned out from the original mesh when generating the base mesh, a mesh with the same number of vertices as the original mesh is obtained. In this specification, these added vertices are also referred to as division points.

[0038] However, when generating the base mesh, i.e., when thinning out vertices from the original mesh, the connections are updated. Therefore, division points are formed on these updated connections. Therefore, the shape of this subdivided base mesh may not match the shape of the original mesh. More specifically, as shown in the bottom part of Figure 2, the positions of the division points (on the dotted lines) may not match the original mesh. In this specification, the difference between the positions of the vertices (division points) of this subdivided base mesh and the positions of the vertices of the original mesh is referred to as a displacement vector. In other words, the displacement vector indicates the difference between the positions of corresponding vertices between the subdivided base mesh and the original mesh. In other words, the original mesh is obtained by updating the positions of each vertex of the subdivided base mesh according to this displacement vector.

[0039] In other words, by utilizing such a conversion process, ideally, the original mesh can be obtained at the destination by transmitting the base mesh and the displacement vector. In other words, transmitting the base mesh and the displacement vector can replace the transmission of the original mesh. When transmitting data, it is common to encode the data and transmit it, and then decode the encoded data at the destination to generate (restore) the data. Since the original mesh is known to the encoder (transmitting device), it can generate the base mesh and further derive the displacement vector. In other words, the encoder can encode the base mesh and the displacement vector. The decoder (transmitting device) can decode the transmitted encoded data and generate (restore) the base mesh and the displacement vector. The decoder can then subdivide the base mesh and apply the displacement vector to each vertex to generate (restore) the original mesh. Note that in this specification, the original mesh generated by the decoder is also referred to as the "decoded original mesh." In contrast, the original mesh before being encoded by the encoder is also referred to as the "original mesh before encoding." The original mesh after decoding is a mesh that represents the three-dimensional structure of an object whose original mesh before encoding represents a three-dimensional structure, and can be said to be substantially identical to the original mesh before encoding. However, in reality, the original mesh after decoding may have different polygon configurations, and therefore may not be completely identical in terms of information to the original mesh before encoding.

[0040] As described above, the base mesh is coarser than the original mesh, and therefore the amount of data is smaller than that of the original mesh. Furthermore, while vertex positions in the original mesh are expressed as absolute values, the displacement vectors represent differential information between vertex positions, and therefore the amount of data is smaller than that of the original mesh. Furthermore, by storing the displacement vectors in a two-dimensional image and encoding them using 2D encoding, encoding efficiency can be improved. Therefore, the amount of code required for encoding the base mesh and displacement vectors can be reduced compared to encoding the original mesh. In other words, encoding and decoding the base mesh and displacement vectors using the above-described conversion process can improve encoding efficiency compared to encoding and decoding the original mesh. In other words, transmitting the base mesh and displacement vectors using the above-described conversion process can reduce the amount of data transmission and improve transmission efficiency compared to transmitting the original mesh. Note that, in this specification, storing data in a two-dimensional image is also referred to as packing.

[0041] In this specification, something that changes in the time direction is also referred to as "dynamic." In addition, something that remains unchanged in the time direction or has no concept of time direction is also referred to as "static." For example, a mesh representing the three-dimensional shape of an object at one time can be said to be static. When a sequence consists of multiple meshes representing the three-dimensional shapes of an object at different times, the meshes in the sequence can be said to be dynamic. In other words, a dynamic mesh is made up of static meshes at multiple times. In this specification, information at each time is also referred to as a frame. In other words, a dynamic mesh is made up of meshes of multiple frames.

[0042] In this specification, the three-dimensional shape of an object is dynamic, and the mesh, which is the 3D data of the object, is also dynamic. In other words, the object to be coded is a dynamic mesh. In other words, the base mesh and displacement vectors are also dynamic, and the displacement vectors are coded as a moving image with the two-dimensional images as frames. In this specification, this moving image is also referred to as a displacement video.

[0043] The displacement vectors may be packed into a two-dimensional image as transform coefficients by coefficient transformation such as wavelet transform. The displacement vectors may also be quantized. For example, the displacement vectors may be transformed into transform coefficients by wavelet transform, the transform coefficients may be quantized, and the quantized transform coefficients (quantized coefficients) may be packed.

[0044] <Redundancy in coding of displacement vectors> There are two methods for coding base meshes: intra-coding, which codes each frame independently, and inter-coding, which codes by utilizing the correlation between frames. Generally, inter-coding has higher coding efficiency than intra-coding.

[0045] In contrast, the displacement vectors (displacement video) are transmitted every frame regardless of whether the base mesh is intra-coded or inter-coded.

[0046] For example, the transmission of a current frame and a reference frame as shown in FIG. 3 will be described. The current frame is the frame to be processed. The reference frame is the frame referenced by the current frame. For example, assume that the base mesh of the reference frame is intra-coded. In this case, the base mesh and displacement vector of the reference frame are each coded and transmitted. In this specification, the base mesh of the reference frame is also referred to as a reference base mesh. The displacement vector of the reference frame is also referred to as a reference displacement vector. At the transmission destination, the coded data is decoded to generate (restore) the reference base mesh and reference displacement vector. The reference base mesh is then subdivided. A mesh is reconstructed using the subdivided reference base mesh and reference displacement vector. In this specification, the mesh of the reference frame reconstructed using the reference base mesh and reference displacement vector is also referred to as a reference mesh. In other words, the reference displacement vector indicates the difference in the positions of corresponding vertices between the subdivided reference base mesh and the reference mesh. For example, the reference base mesh 51 shown in the upper part of FIG. 4 is subdivided to generate a subdivided reference base mesh 52. Then, a reference displacement vector (dotted arrow) is applied to each vertex of the subdivided reference base mesh 53 (the same mesh as the subdivided reference base mesh 52), and a reference mesh 54 is reconstructed.

[0047] Assume that the base mesh of the current frame shown in FIG. 3 is inter-coded. In this case, a motion vector and a displacement vector are each coded and transmitted. In this specification, the base mesh of the current frame is also referred to as a current base mesh. The displacement vector of the current frame is also referred to as a current displacement vector. A motion vector is information indicating the positional difference between a vertex of a reference base mesh and a vertex of a current base mesh. At the transmission destination, the coded data is decoded to generate (restore) a motion vector and a current displacement vector. The motion vector is then applied to the reference base mesh to reconstruct a current base mesh. The current base mesh is then subdivided. A mesh is reconstructed using the subdivided current base mesh and the current displacement vector. In this specification, the mesh of the current frame reconstructed using the current base mesh and the current displacement vector is also referred to as a current mesh. In other words, the current displacement vector indicates the difference in position between corresponding vertices of the subdivided current base mesh and the current mesh. For example, a motion vector (dotted arrow) is applied to a reference base mesh 61 shown in the lower part of Figure 4 to reconstruct a current base mesh 62. The current base mesh 62 is then subdivided to generate a subdivided current base mesh 63. Then, a current displacement vector (solid arrow) is applied to each vertex of the subdivided current base mesh 64 (the same mesh as the subdivided current base mesh 63) to reconstruct a current mesh 65.

[0048] In this way, the displacement vector is transmitted for each frame and used to reconstruct the mesh for each frame. However, as shown in Figure 4, this displacement vector is the difference in vertex positions between the subdivided base mesh and the original mesh, and is therefore a parameter that represents the details of the mesh (3D model). Therefore, if the shape of the details of the 3D model does not change, the displacement vector does not change, and transmission of the displacement vector for each frame could become redundant.

[0049] For example, when a person moves their entire arm, the shape of the details of the arm (such as the skin or clothing) may not change (the position of the entire arm may change, but the relative positions of the vertices that make up the arm may not change). In such cases, there may be no or very little change in the time direction of the displacement vector. In other words, there is a risk that transmission of the displacement vector may reduce coding efficiency.

[0050] In the method described in Non-Patent Document 1 and Non-Patent Document 2, since a displacement vector is transmitted, it is difficult to suppress a decrease in coding efficiency in such a case. As described above, the various methods proposed so far do not always achieve the best coding efficiency, and other methods have been desired.

[0051] <3. Reducing Current Displacement Vector Transmission Using Reference Displacement Vector> <Method 1> Therefore, as shown in the top row of the table in Fig. 5, a reference displacement vector is used to reconstruct the current mesh (Method 1). That is, as explained with reference to Fig. 3, in the conventional case, the current displacement vector is applied to reconstruct the current mesh, but in this method, the reference displacement vector of the reference frame is applied instead of the current displacement vector.

[0052] When there is no (or very little) change in the shape of the details, it can be estimated that the difference between the reference displacement vector and the current displacement vector is also small. Therefore, by applying the transmitted reference displacement vector instead of the current displacement vector, the transmission of the current displacement vector can be skipped (omitted). In other words, the amount of transmitted displacement vectors can be reduced. Therefore, the reduction in coding efficiency can be suppressed.

[0053] <Method 1-1> When Method 1 is applied, for example, a reference displacement vector may be applied to a current base mesh (Method 1-1), as shown in the second row from the top of the table in FIG. 5 . For example, as shown in FIG. 6 , in the current frame, the current base mesh is reconstructed by applying a motion vector to the vertices of a reference base mesh. Then, the current base mesh is subdivided. Then, the reference displacement vector of the reference frame is applied to the subdivided current base mesh instead of the current displacement vector, thereby reconstructing the current mesh. For example, as shown in FIG. 7 , a current base mesh 112 is reconstructed by applying a motion vector (dotted arrow) to the vertices of a reference base mesh 111. Next, the current base mesh 112 is subdivided to generate a subdivided current base mesh 113. Then, a reference displacement vector (dotted arrow) is applied to each vertex (each division point) of a subdivided current base mesh 114 (the same mesh as the current base mesh 113) to reconstruct a current mesh 115.

[0054] By doing so, it is possible to skip (omit) the transmission of the current displacement vector for reconstructing the current mesh. In other words, it is possible to reduce the amount of transmission of the displacement vector. Therefore, it is possible to suppress a decrease in coding efficiency.

[0055] Note that the reference displacement vector may be applied to all vertices (all division points) of the current base mesh. For example, in the case of the subdivided current base mesh 114 of FIG. 7 , the reference displacement vector (dotted arrow) is applied to all division points. Alternatively, the reference displacement vector may be applied to some of the vertices (some division points) of the current base mesh. In other words, in this case, the current displacement vector is applied to the remaining vertices (remaining division points). For example, in the case of the subdivided current base mesh 116 (the same mesh as the current base mesh 113) of FIG. 7 , the reference displacement vector (dotted arrow) is applied to some division points, and the current displacement vector (solid arrow) is applied to the remaining division points. For example, flexible application can be achieved by applying the current displacement vector to locations where the displacement vector changes significantly in the time direction and applying the reference displacement vector to locations where the displacement vector changes little in the time direction. In other words, the current displacement vector may be applied as needed. This allows for comprehensive control, such as suppressing a decrease in encoding efficiency while suppressing a decrease in the subjective quality of the mesh.

[0056] When and where to apply the reference displacement vector to the vertices of the current base mesh in the reconstruction of the current mesh may be determined in any manner. For example, it may be predetermined. Alternatively, the encoder may provide control information to the decoder along with the bitstream to control when and where the reference displacement vector is applied to the reconstruction of the current mesh. That is, the decoder may apply the reference displacement vector in accordance with the control information.

[0057] Such control information regarding the application of the reference displacement vectors to the vertices of the current base mesh may include any content.

[0058] For example, the control information may include parameters specifying the target and range of reuse (i.e., information indicating where to apply the reference displacement vector). That is, the encoder may provide the control information including such parameters to the decoder. The decoder may also apply the reference displacement vector to the division points of the current base mesh that are specified as targets by the parameters or to division points within the range specified by the parameters.

[0059] For example, the parameter may be flag information for each sequence. For example, if the value of the flag information is true (e.g., 1), the reference displacement vector may be applied to the vertices of the current base mesh in all interframes (frames to which inter-coding is applied) in the sequence. Alternatively, an enable flag may be provided. That is, the encoder may provide control information including such flag information to the decoder. The decoder may then determine whether to apply the reference displacement vector according to the flag information.

[0060] The flag information may also be information on a frame-by-frame basis. For example, the flag information may be provided for each frame, and the decoder may apply the reference displacement vector to the vertices of the current base mesh in frames where the value of the flag information is true (e.g., 1). Also, a list of identification information of frames (frame ID list) to which the reference displacement vector is applied to the vertices of the current base mesh may be provided on a sequence-by-sequence basis.

[0061] This control may also be performed for each small region within a frame. For example, the control information may include parameters specifying the small region to which the reference displacement vector is applied to the vertices of the current base mesh. For example, the small region to which the reference displacement vector is applied to the vertices of the current base mesh may be specified for each frame. That is, the encoder may provide control information including such parameters to the decoder. Alternatively, the decoder may apply the reference displacement vector to the division point of the small region of the current base mesh specified by the parameters. Note that the information unit (small region) for which this control is performed is arbitrary. For example, this control may be performed for each submesh, each face, or each level of detail (LoD), or a combination of controls for multiple information units may be applied. In either case, the relevant location may be indicated using identification information for the information unit, etc. Note that the target or range may be specified using a conditional expression, etc.

[0062] The control information may also include a parameter specifying a reference target. That is, it may specify which frame is to be used as a key frame (a frame that can be used as a reference frame) for the displacement vector. That is, the encoder may provide the control information including such a parameter to the decoder. The decoder may also use the key frame specified by the parameter as a reference frame. In other words, it may specify in which frame the displacement vector is to be transmitted.

[0063] Any method may be used to specify the key frames (intraframes) for the displacement vectors. For example, a list of key frames (intraframes) for the displacement vectors may be included in the control information. When a frame other than a key frame is set as the current frame, the displacement vector of the closest key frame may be used as a reference displacement vector for reconstructing the mesh.

[0064] Alternatively, a reference frame may be specified for each frame. For example, a list of identification information of reference frames for each frame may be included in the control information. In this case, the reference frame for each frame can be set arbitrarily. That is, the encoder may provide control information including the list to the decoder. Alternatively, the decoder may use the frame specified by the list as the reference frame.

[0065] In this way, the key frames for the displacement vectors may be set independently of the key frames for the base mesh. For example, in Fig. 8, the horizontal axis indicates the sequence (frame direction), the vertical lines indicate each frame, and the thick lines indicate key frames. As shown in Fig. 8, frames different from the key frames for the base mesh may also be set as key frames for the displacement vectors.

[0066] The control information may also include information specifying the keyframes for the displacement vectors as a percentage of the keyframe interval of the base mesh. For example, the displacement vectors may be transmitted at intervals twice the keyframe interval of the base mesh.

[0067] Of course, the reference frame may be unified with that of the base mesh, in which case this parameter may be omitted.

[0068] <Encoding device> Fig. 9 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology (the above-mentioned method 1-1) is applied. The encoding device 200 shown in Fig. 9 is a device that encodes a dynamic mesh. The encoding device 200 acquires a current mesh of the dynamic mesh, performs inter-encoding on it, generates a bitstream, and outputs it.

[0069] Note that Fig. 9 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 9. In other words, in the encoding device 200, there may be processing units that are not shown as blocks in Fig. 9, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 9.

[0070] 9, the encoding device 200 includes a control unit 201, a transmission data generation unit 202, and a bitstream generation unit 203. The bitstream generation unit 203 includes a motion vector encoding unit 211, a displacement video encoding unit 212, a control information encoding unit 213, and a multiplexing unit 214.

[0071] The control unit 201 controls each processing unit in the encoding device 200 (each processing unit in the transmission data generation unit 202 and the bitstream generation unit 203) to execute processing for applying a reference displacement vector to a current base mesh on the decoder side. For example, the control unit 201 controls the transmission data generation unit 202 so that it does not encode the current displacement vector (or only partially encodes it) in frames other than key frames for the displacement vector. For example, the control unit 201 may control the transmission data generation unit 202 so that it does not generate displacement vectors (or only partially encodes them). The other processing units operate under the control of the control unit 201. The control unit 201 may also generate control information related to the application of the reference displacement vector to the vertices of the current base mesh. This control information is transmitted to the decoder and used by the decoder. Therefore, the control unit 201 may supply the generated control information to the control information encoding unit 213.

[0072] The transmission data generation unit 202 converts the supplied mesh into a format for transmission. For example, the transmission data generation unit 202 may generate a current base mesh from the supplied current mesh and supply it to the motion vector encoding unit 211. The transmission data generation unit 202 may also skip (omit) generation of a current displacement vector under the control of the control unit 201. The transmission data generation unit 202 may also generate a current displacement vector for a part of the mesh under the control of the control unit 201 and supply it to the displacement video encoding unit 212.

[0073] The bitstream generation unit 203 performs processing related to bitstream generation. The motion vector encoding unit 211 obtains a reference base mesh and a current base mesh and generates a motion vector using them. In other words, the motion vector encoding unit 211 uses the difference in vertex positions between the reference base mesh and the current base mesh as a motion vector. The motion vector encoding unit 211 encodes the motion vector and supplies the encoded data to the multiplexing unit 214.

[0074] The displacement video encoding unit 212 packs the current displacement vector (part) supplied from the transmission data generation unit 202 into a displacement video frame and encodes it as a displacement video. The displacement video encoding unit 212 supplies the encoded data to the multiplexing unit 214.

[0075] The control information encoding unit 213 acquires and encodes the control information supplied from the control unit 201 , and supplies the encoded data to the multiplexing unit 214 .

[0076] The multiplexing unit 214 acquires the coded data of the motion vector supplied from the motion vector coding unit 211, the coded data of the displacement video supplied from the displacement video coding unit 212, and the coded data of the control information supplied from the control information coding unit 213, and multiplexes these coded data to generate a bitstream. The multiplexing unit 214 outputs the generated bitstream to the outside of the coding device 200. This bitstream is transmitted to a decoder (a decoding device 250 described later) via an arbitrary storage medium, communication medium, etc.

[0077] The present technology (method 1-1 described above) may be applied to the encoding device 200 configured as described above. For example, the motion vector encoding unit 211 of the encoding device 200 may encode a motion vector indicating a difference in position between a vertex of a reference base mesh and a vertex of a current base mesh. Furthermore, the control information encoding unit 213 may encode control information related to application of a reference displacement vector to a vertex of a current base mesh when reconstructing the current mesh.

[0078] With this configuration, the encoding device 200 can control the application of reference displacement vectors to the vertices of the current base mesh when the decoder reconstructs the current mesh, thereby preventing a decrease in encoding efficiency.

[0079] As described above, the encoding device 200 may include a displacement video encoding unit 212. The displacement video encoding unit 212 may encode a current displacement vector indicating the difference in position between some vertices of the current mesh and the vertices of the current base mesh. In other words, the displacement video encoding unit 212 may also be referred to as a current displacement vector encoding unit.

[0080] <Flow of Inter-Encoding Process> The encoding device 200 in Fig. 9 performs inter-encoding on the current mesh. An example of the flow of this inter-encoding process will be described with reference to the flowchart in Fig. 10 .

[0081] When the inter-encoding process starts, in step S201, the transmission data generation unit 202 generates current transmission data (such as a current base mesh) using the current mesh.

[0082] In step S202, the motion vector encoding unit 211 generates a motion vector using the reference base mesh and the current base mesh, and encodes the motion vector.

[0083] In step S203, the control unit 201 determines whether or not to transmit the current displacement vector of a part of the current mesh. If it is determined that the current displacement vector is to be transmitted, the process proceeds to step S204.

[0084] In step S204, the displacement video encoding unit 212 generates a displacement video using a current displacement vector of a part of the current mesh and encodes the displacement video. When the process of step S204 is completed, the process proceeds to step S205. Also, if it is determined in step S203 that the current displacement vector is not to be transmitted, the process of step S204 is skipped (omitted), and the process proceeds to step S205.

[0085] In step S205, the control unit 201 generates control information related to application of the reference displacement vector to the vertices of the current base mesh in the reconstruction of the current mesh. In step S206, the control information encoding unit 213 encodes the control information.

[0086] In step S207, the multiplexing unit 214 multiplexes the coded data of the motion vectors and the coded data of the control information (and also the coded data of the displacement video (part of the displacement vectors) if the displacement video is coded), and generates a bitstream. When the process of step S207 is completed, the inter-coding process is completed.

[0087] In such an inter-coding process, the present technology (method 1-1 described above) may be applied. For example, in step S202, the motion vector encoding unit 211 may encode a motion vector indicating a difference in position between a vertex of a reference base mesh and a vertex of a current base mesh. In addition, in step S206, the control information encoding unit 213 may encode control information regarding application of a reference displacement vector to a vertex of the current base mesh in the reconstruction of the current mesh.

[0088] By performing the inter-coding process in this manner, the encoding device 200 can control the application of reference displacement vectors to the vertices of the current base mesh when the decoder reconstructs the current mesh, thereby preventing a decrease in encoding efficiency.

[0089] <Decoding device> Fig. 11 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which the present technology (the above-described method 1) is applied. The decoding device 250 shown in Fig. 11 is a device that performs inter-decoding of coded data (bit stream) to generate (reconstruct) a dynamic mesh. The decoding device 250 acquires the bit stream, decodes it, generates a dynamic mesh, and outputs it.

[0090] Note that Fig. 11 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in the decoding device 250, there may be processing units that are not shown as blocks in Fig. 11, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 11.

[0091] 11 , the decoding device 250 includes a demultiplexing unit 251, a decoding unit 252, a control unit 253, and a reconstruction unit 254. The decoding unit 252 includes a control information decoding unit 261, a motion vector decoding unit 262, and a displacement video decoding unit 263. The reconstruction unit 254 includes a current base mesh reconstruction unit 271 and a current mesh reconstruction unit 272.

[0092] The demultiplexing unit 251 acquires a bitstream supplied from outside the decoding device 250 (e.g., from the encoding device 200). The demultiplexing unit 251 demultiplexes this bitstream to extract coded data of control information and coded data of motion vectors. Furthermore, if coded data of displacement video is included in the bitstream, the demultiplexing unit 251 also extracts the coded data of the displacement video.

[0093] The decoding unit 252 performs processing related to decoding. The control information decoding unit 261 acquires coded data of control information supplied from the demultiplexing unit 251. The control information decoding unit 261 decodes the coded data and generates (restores) control information. The control information decoding unit 261 supplies the control information to the control unit 253. The motion vector decoding unit 262 acquires coded data of motion vectors supplied from the demultiplexing unit 251. The motion vector decoding unit 262 decodes the coded data and generates (restores) motion vectors. The motion vector decoding unit 262 supplies the motion vectors to the current base mesh reconstruction unit 271. The displacement video decoding unit 263 acquires coded data of displacement video when it is supplied from the demultiplexing unit 251. The displacement video decoding unit 263 decodes the coded data and generates (restores) displacement video. The displacement video decoding unit 263 unpacks the displacement video to extract the current displacement vector. The displacement video decoder 263 supplies the current displacement vector to the current mesh reconstructor 272 .

[0094] The control unit 253 controls each processing unit in the decoding device 250 based on the control information supplied from the control information decoding unit 261, and causes the processing unit to execute processing related to application of the reference displacement vector in the reconstruction of the current mesh.

[0095] The reconstruction unit 254 performs processing related to mesh reconstruction. The current base mesh reconstruction unit 271 applies the motion vectors supplied from the motion vector decoding unit 262 to the vertices of the reference base mesh to reconstruct the current base mesh. The current base mesh reconstruction unit 271 supplies the reconstructed current base mesh to the current mesh reconstruction unit 272. The current mesh reconstruction unit 272 applies the reference displacement vectors to the vertices of the current base mesh to reconstruct the current mesh. Note that when a current displacement vector for a portion of the mesh is supplied from the displacement video decoding unit 263, the current mesh reconstruction unit 227 reconstructs the current mesh by applying the current displacement vector to the vertices of that portion of the current base mesh and the reference displacement vector to the vertices of the remaining portion. The current mesh reconstruction unit 272 outputs the reconstructed current mesh to an external device of the decoding device 250.

[0096] The present technology (method 1-1 described above) may be applied to a decoding device 250 configured in this manner. For example, the motion vector decoding unit 262 of the decoding device 250 may decode encoded data and generate motion vectors indicating the positional differences between the vertices of the reference base mesh and the vertices of the current base mesh. Furthermore, the current base mesh reconstruction unit 271 may reconstruct the current base mesh by applying the motion vectors to the vertices of the reference base mesh. Furthermore, the current mesh reconstruction unit 272 may reconstruct the current mesh by applying the reference displacement vectors to the vertices of the current base mesh.

[0097] With this configuration, the decoding device 250 can apply the reference displacement vector to the vertices of the current base mesh in the reconstruction of the current mesh, thereby suppressing a decrease in coding efficiency.

[0098] The current mesh reconstruction unit 272 may apply the reference displacement vector to all of the vertices generated by subdividing the current base mesh. Alternatively, the current mesh reconstruction unit 272 may apply the reference displacement vector to some of the vertices generated by subdividing the current base mesh, and apply a current displacement vector indicating the difference in position between the vertices obtained by subdividing the current base mesh and the vertices of the current mesh to the remaining vertices.

[0099] As described above, the decoding device 250 may also include a control information decoder 261 and a controller 253. The control information decoder 261 may decode the encoded data and generate control information related to the application of reference displacement vectors to the vertices of the current base mesh. The controller 253 may also control the reconstruction of the current mesh based on the control information.

[0100] <Flow of Inter-Decoding Process> The decoding device 250 in Fig. 11 performs inter-decoding on a bitstream. An example of the flow of this inter-decoding process will be described with reference to the flowchart in Fig. 12.

[0101] When the inter-decoding process starts, the demultiplexing unit 251 demultiplexes the bitstream in step S251 to extract coded data of control information and coded data of motion vectors. Furthermore, if coded data of displacement video is included in the bitstream, the demultiplexing unit 251 also extracts the coded data of the displacement video. In step S252, the control information decoding unit 261 decodes the coded data of the control information extracted from the bitstream to generate (restore) the control information. In step S253, the motion vector decoding unit 262 decodes the coded data of the motion vector extracted from the bitstream to generate (restore) the motion vector. In step S254, the current base mesh reconstruction unit 271 applies the motion vector to the vertices of the reference base mesh to reconstruct the current base mesh.

[0102] In step S255, the control unit 253 determines whether a current displacement vector exists. If it is determined that a current displacement vector exists (is supplied from the encoding device 200), the process proceeds to step S256. In step S256, the displacement video decoding unit 263 decodes the encoded data of the displacement video to generate (restore) the displacement video. The displacement video decoding unit 263 unpacks and extracts the current displacement vector from the displacement video.

[0103] In step S257, the current mesh reconstructing unit 272 reconstructs the current mesh using the current base mesh, the reference displacement vector, and the current displacement vector. When the process of step S257 ends, the inter-decoding process ends.

[0104] If it is determined in step S255 that the current displacement vector does not exist, the process proceeds to step S258. In step S258, the current mesh reconstruction unit 272 reconstructs the current mesh using the current base mesh and the reference displacement vector. When the process of step S258 ends, the inter-decoding process ends.

[0105] The present technology (method 1-1 described above) may be applied to such inter-decoding processing. For example, in step S253, the motion vector decoding unit 262 may decode the encoded data and generate motion vectors indicating the positional differences between the vertices of the reference base mesh and the vertices of the current base mesh. Also, in step S254, the current base mesh reconstruction unit 271 may reconstruct the current base mesh by applying the motion vectors to the vertices of the reference base mesh. Also, in step S257 or step S258, the current mesh reconstruction unit 272 may reconstruct the current mesh by applying the reference displacement vectors to the vertices of the current base mesh.

[0106] By performing the inter-decoding process in this manner, the decoding device 250 can apply the reference displacement vector to the vertices of the current base mesh in the reconstruction of the current mesh, thereby preventing a decrease in coding efficiency.

[0107] <Method 1-1-1> When Method 1-1 is applied, for example, as shown in the third row from the top of the table in FIG. 5 , a current displacement vector may be further applied to a current base mesh to which a reference displacement vector has been applied (Method 1-1-1). For example, as shown in FIG. 13 , a current displacement vector may be further applied to a mesh (i.e., vertices to which a reference displacement vector has been applied) reconstructed in the current frame in the same manner as in the example in FIG. 6 . For example, as shown in FIG. 14 , a reference displacement vector (dotted arrow) is applied to each vertex of a subdivided current mesh 301 to reconstruct a mesh 302, and a current displacement vector (solid arrow) is further applied to each vertex of the mesh 302 to reconstruct a current mesh 303. In other words, in this case, as shown in FIG. 14 , both a reference displacement vector and a current displacement vector are applied to one vertex.

[0108] When a reference displacement vector is applied to the reconstruction of the current mesh, unless the displacement vectors in the current frame and the reference frame are completely identical, theoretically, the subjective quality of the mesh will be reduced. On the other hand, transmitting the displacement vector every frame increases the amount of coding. Therefore, a reference displacement vector is applied to the reconstruction of the current mesh, and the difference between the displacement vectors in the current frame and the reference frame is used as the current displacement vector. In this way, the amount of information in the current displacement vector to be transmitted can be reduced, and the difference between the displacement vectors in the current frame and the reference frame can be reduced. Therefore, it is possible to suppress a reduction in the subjective quality of the mesh while suppressing a reduction in encoding efficiency.

[0109] <Encoding device> Fig. 15 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology (the above-described method 1-1-1) is applied. The encoding device 400 shown in Fig. 15 is a device similar to the encoding device 200, and encodes a dynamic mesh. That is, the encoding device 400 obtains a current mesh of the dynamic mesh, performs inter-encoding, generates a bitstream, and outputs it.

[0110] Note that Fig. 15 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 15. In other words, in encoding device 400, there may be processing units that are not shown as blocks in Fig. 15, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 15.

[0111] 15 , the encoding device 400 includes a control unit 401, a transmission data generation unit 402, and a bitstream generation unit 403. The bitstream generation unit 403 includes a motion vector encoding unit 411, a current displacement vector derivation unit 412, a displacement video encoding unit 413, a control information encoding unit 414, and a multiplexing unit 415.

[0112] The control unit 401 is a processing unit similar to the control unit 201 (FIG. 9) and performs similar processing. The transmission data generation unit 402 is a processing unit similar to the transmission data generation unit 202 (FIG. 9) and performs similar processing. The motion vector coding unit 411 is a processing unit similar to the motion vector coding unit 211 (FIG. 9) and performs similar processing.

[0113] The current displacement vector derivation unit 412 acquires the current base mesh and the current mesh supplied from the transmission data generation unit 402. The current displacement vector derivation unit 412 also acquires a reference displacement vector. The current displacement vector derivation unit 412 compares the current base mesh to which the reference displacement vector has been applied to the vertices of the current base mesh with the current mesh, and determines the difference (the difference in the positions of each vertex) as a current displacement vector for additional correction and supplies it to the displacement video encoding unit 413.

[0114] The displacement video encoding unit 413 packs the current displacement vector for additional correction supplied from the current displacement vector derivation unit 412 into the displacement video and encodes it. The displacement video encoding unit 413 supplies the encoded data of the displacement video to the multiplexing unit 415.

[0115] The control information encoding unit 414 is a processing unit similar to the control information encoding unit 213 ( FIG. 9 ) and performs similar processing. As with the multiplexing unit 214 ( FIG. 9 ), the multiplexing unit 415 acquires the coded data of the motion vector supplied from the motion vector encoding unit 411, the coded data of the displacement video supplied from the displacement video encoding unit 413, and the coded data of the control information supplied from the control information encoding unit 414, and multiplexes these coded data to generate a bitstream. However, the coded data of the displacement video has the current displacement vector for additional correction described above packed therein.

[0116] The present technology (the above-described method 1-1-1) may be applied to an encoding device 400 configured as described above. For example, the encoding device 400 may have a current displacement vector derivation unit 412 and a displacement video encoding unit 413, as described above. The current displacement vector derivation unit 412 may derive a current displacement vector indicating a difference in position between a vertex of a current mesh and a vertex of a mesh to which a reference displacement vector has been applied to the vertex of a current base mesh. The displacement video encoding unit 413 may also encode the current displacement vector. In other words, the displacement video encoding unit 413 may also be referred to as a current displacement vector encoding unit.

[0117] With this configuration, the encoding device 400 can provide the decoder with a current displacement vector for additional correction, thereby suppressing a decrease in encoding efficiency while suppressing a decrease in subjective quality of the mesh.

[0118] <Flow of Inter-Encoding Process> The encoding device 400 in Fig. 15 performs inter-encoding on the current mesh. An example of the flow of this inter-encoding process will be described with reference to the flowchart in Fig. 16 .

[0119] The processes in steps S401 and S402 are executed in the same manner as the processes in steps S201 and S202 (FIG. 10).

[0120] In step S403, the current displacement vector derivation unit 412 derives a current displacement vector for additional correction. In step S404, the displacement video encoding unit 413 generates a displacement video using the current displacement vector for additional correction and encodes the displacement video.

[0121] The processes of steps S405 to S407 are performed in the same manner as the processes of steps S205 to S207 (FIG. 10). However, the coded data of the disparity video includes the current disparity vector for additional correction. When the process of step S407 is completed, the inter-coding process is completed.

[0122] In such an inter-encoding process, the present technology (the above-described method 1-1-1) may be applied. For example, in step S403, the current displacement vector derivation unit 412 may derive a current displacement vector (current displacement vector for additional correction) indicating a difference in position between a vertex of the current mesh and a vertex of a mesh to which a reference displacement vector has been applied to a vertex of the current base mesh. Furthermore, in step S404, the displacement video encoding unit 413 may pack the current displacement vector for additional correction into a displacement video and encode the displacement video.

[0123] By performing the inter-coding process in this manner, the encoding device 400 can provide the decoder with a current displacement vector for additional correction, thereby suppressing a decrease in encoding efficiency while suppressing a decrease in the subjective quality of the mesh.

[0124] <Decoding device> Fig. 17 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which the present technology (the above-described method 1-1-1) is applied. The decoding device 450 shown in Fig. 17 is a device similar to the decoding device 250, and performs inter-decoding of encoded data (bit stream) to generate (reconstruct) a dynamic mesh. In other words, the decoding device 450 obtains the bit stream, decodes it, generates a dynamic mesh, and outputs it.

[0125] Note that Fig. 17 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in the decoding device 450, there may be processing units that are not shown as blocks in Fig. 17, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 17.

[0126] 17 , the decoding device 450 includes a demultiplexing unit 451, a decoding unit 452, a control unit 453, and a reconstruction unit 454. The decoding unit 452 includes a control information decoding unit 461, a motion vector decoding unit 462, and a displacement video decoding unit 463. The reconstruction unit 454 includes a current base mesh reconstruction unit 471, a current mesh reconstruction unit 472, and an additional correction unit 473.

[0127] The demultiplexing unit 451 is a processing unit similar to the demultiplexing unit 251 ( FIG. 11 ) and performs similar processing. The control unit 453 is a processing unit similar to the control unit 253 ( FIG. 11 ) and performs similar processing. The control information decoding unit 461 is a processing unit similar to the control information decoding unit 261 ( FIG. 11 ) and performs similar processing. The motion vector decoding unit 462 is a processing unit similar to the motion vector decoding unit 262 ( FIG. 11 ) and performs similar processing. The displacement video decoding unit 463 is a processing unit similar to the displacement video decoding unit 463 ( FIG. 11 ) and performs similar processing. The current base mesh reconstruction unit 471 is a processing unit similar to the current base mesh reconstruction unit 271 ( FIG. 11 ) and performs similar processing. The current mesh reconstruction unit 472 is a processing unit similar to the current mesh reconstruction unit 272 ( FIG. 11 ) and performs similar processing.

[0128] However, the dislocation video is packed with a current dislocation vector for additional correction, and the dislocation video decoding unit 463 extracts the current dislocation vector for additional correction and supplies it to the additional correction unit 473. In addition, the current mesh reconstruction unit 472 supplies the reconstructed current mesh to the additional correction unit 473.

[0129] The additional correction unit 473 applies the current displacement vector for additional correction to the vertices of the current mesh supplied thereto, and outputs the current mesh after the additional correction to the outside of the decoding device 450.

[0130] The present technology (the above-described method 1-1-1) may be applied to a decoding device 450 configured in this manner. For example, the decoding device 450 may include an additional correction unit 473 as described above. The additional correction unit 473 may then apply, to the vertices of the current mesh, a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices obtained by further subdividing the current base mesh (i.e., a current displacement vector for additional correction). Therefore, the additional correction unit 473 can also be said to be a displacement vector application unit.

[0131] With this configuration, the decoding device 450 can apply the supplied current displacement vector for additional correction to the current mesh, thereby suppressing a decrease in encoding efficiency while suppressing a decrease in the subjective quality of the mesh.

[0132] <Flow of Inter-Decoding Process> The decoding device 450 in Fig. 17 performs inter-decoding on a bitstream. An example of the flow of this inter-decoding process will be described with reference to the flowchart in Fig. 18 .

[0133] The processes of steps S451 to S454 are executed in the same manner as the processes of steps S251 and S254 (FIG. 12). The process of step S455 is executed in the same manner as the process of step S258 (FIG. 12).

[0134] In step S456, the displaced video decoding unit 463 decodes the encoded data of the displaced video to generate the displaced video, and unpacks the displaced video to extract the current displaced vector for additional correction.

[0135] In step S457, the additional correction unit 473 performs additional correction on the current mesh using the current displacement vector for additional correction. That is, the additional correction unit 473 applies the current displacement vector for additional correction to the vertices of the current mesh. When the processing of step S457 ends, the inter-decoding processing ends.

[0136] In such inter-decoding processing, the present technology (the above-described method 1-1-1) may be applied. For example, in step S457, the additional correction unit 473 may further apply, to the vertices of the current mesh, a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices obtained by further subdividing the current base mesh (i.e., a current displacement vector for additional correction).

[0137] By performing the inter-decoding process in this manner, the decoding device 450 can apply the supplied current displacement vector for additional correction to the current mesh, thereby suppressing a decrease in encoding efficiency while suppressing a decrease in the subjective quality of the mesh.

[0138] <Method 1-2> When Method 1 is applied, for example, as shown in the fourth row from the top of the table in FIG. 5 , the reference mesh may be reused and a current displacement vector may be applied (Method 1-2). For example, as shown in FIG. 19 , the reference mesh reconstructed in the current frame is reused as the current mesh in the current frame, and a current displacement vector may be applied. For example, as shown in FIG. 20 , a reference displacement vector (dotted arrow) is applied to the vertices of a subdivided reference base mesh 501, thereby reconstructing a reference mesh 502. A current displacement vector (solid arrow) is then applied to the vertices of the reference mesh 502, thereby reconstructing a current mesh 503. Therefore, as shown in FIG. 20 , both the reference displacement vector and the current displacement vector are applied to one vertex.

[0139] Therefore, as in the case of the above-mentioned method 1-1-1, it is possible to reduce the amount of information of the current displacement vector to be transmitted and to reduce the difference between the displacement vectors of the current frame and the reference frame, thereby suppressing the reduction in the coding efficiency while suppressing the reduction in the subjective quality of the mesh.

[0140] In this method 1-2, as described above, the reference mesh is reused as the current mesh. Therefore, there is no need to reconstruct the base mesh in the current frame, and the load of the decoding process can be reduced. However, in order to reduce the degradation of the subjective quality of the mesh, it is desirable to minimize the difference in vertex positions between the reference mesh and the current mesh (i.e., the reference base mesh and the current base mesh).

[0141] In addition, in the case of Method 1-2, by reusing the reference mesh in the current frame, it can be said that the reference displacement vector is used to reconstruct the current mesh, similar to the above-mentioned Method 1 and the like.

[0142] <Encoding device> Fig. 21 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology (the above-mentioned method 1-2) is applied. The encoding device 600 shown in Fig. 21 is a device similar to the encoding device 400, and encodes a dynamic mesh. That is, the encoding device 600 obtains a current mesh of the dynamic mesh, performs inter-encoding, generates a bitstream, and outputs it.

[0143] Note that Fig. 21 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in the encoding device 600, there may be processing units that are not shown as blocks in Fig. 21, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 21.

[0144] 21 , the encoding device 600 includes a control unit 601, a transmission data generation unit 602, and a bitstream generation unit 603. The bitstream generation unit 603 includes a current displacement vector derivation unit 611, a displacement video encoding unit 612, a control information encoding unit 613, and a multiplexing unit 614.

[0145] The control unit 601 is a processing unit similar to the control unit 401 (FIG. 15) and executes the same processing. The transmission data generation unit 602 is a processing unit similar to the transmission data generation unit 402 (FIG. 15) and executes the same processing.

[0146] The current displacement vector derivation unit 611 acquires a current mesh supplied from the transmission data generation unit 602. The current displacement vector derivation unit 611 also acquires a reference mesh. The current displacement vector derivation unit 611 compares the current mesh with the reference mesh, and determines the difference (the difference between the positions of each vertex) as a current displacement vector for additional correction, and supplies this to the displacement video encoding unit 612.

[0147] The displacement video encoding unit 612 is a processing unit similar to the displacement video encoding unit 413 (FIG. 15) and performs similar processing. The control information encoding unit 613 is a processing unit similar to the control information encoding unit 414 (FIG. 15) and performs similar processing. The multiplexing unit 614 is a processing unit similar to the multiplexing unit 415 (FIG. 15) and performs similar processing.

[0148] However, the current displacement vector for additional correction that is acquired by the displacement video encoding unit 612 and packed into the displacement video indicates the difference between the current mesh and the reference mesh (the difference in the positions of each vertex), as described above.

[0149] The present technology (the above-described method 1-2) may be applied to the encoding device 600 configured as described above. For example, the current displacement vector derivation unit 611 of the encoding device 600 may derive a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the reference mesh. Furthermore, the displacement video encoding unit 612 may encode the current displacement vector. Therefore, the displacement video encoding unit 612 can also be referred to as a current displacement vector encoding unit.

[0150] The encoding device 600 may also include a control information encoding unit 613. The control information encoding unit 613 may encode control information related to application of the current displacement vector to the vertices of the reference mesh in the reconstruction of the current mesh.

[0151] With this configuration, the encoding device 600 can provide the decoder with a current displacement vector for additional correction, thereby suppressing a decrease in encoding efficiency while suppressing a decrease in subjective quality of the mesh.

[0152] <Flow of Inter-Encoding Process> The encoding device 600 in Fig. 21 performs inter-encoding on the current mesh. An example of the flow of this inter-encoding process will be described with reference to the flowchart in Fig. 22.

[0153] The process of step S601 is executed in the same manner as the process of step S401 (FIG. 16). In step S602, the current displacement vector derivation unit 611 derives a current displacement vector for additional correction using the reference mesh and the current mesh.

[0154] In step S603, the displaced video encoding unit 612 generates a displaced video using the current displaced vector for additional correction, and encodes the displaced video.

[0155] The processes of steps S604 to S606 are executed in the same manner as the processes of steps S405 to S407. When the process of step S606 ends, the inter-encoding process ends.

[0156] In such an inter-coding process, the present technology (the above-described method 1-2) may be applied. For example, in step S602, the current displacement vector derivation unit 611 may derive a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the reference mesh. Furthermore, in step S603, the displacement video encoding unit 612 may encode the current displacement vector. Therefore, the displacement video encoding unit 612 can also be referred to as a current displacement vector encoding unit.

[0157] By performing the inter-coding process in this manner, the encoding device 600 can provide the decoder with a current displacement vector for additional correction, thereby suppressing a decrease in the subjective quality of the mesh while suppressing a decrease in the encoding efficiency.

[0158] <Decoding device> Fig. 23 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which the present technology (the above-described method 1-2) is applied. The decoding device 650 shown in Fig. 23 is a device similar to the decoding device 450, and performs inter-decoding of coded data (bit stream) to generate (reconstruct) a dynamic mesh. In other words, the decoding device 650 obtains the bit stream, decodes it, generates a dynamic mesh, and outputs it.

[0159] Note that Fig. 23 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in the decoding device 650, there may be processing units that are not shown as blocks in Fig. 23, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 23.

[0160] 23, the decoding device 650 includes a demultiplexing unit 651, a decoding unit 652, a control unit 653, and an additional correction unit 654. The decoding unit 652 includes a control information decoding unit 661 and a displacement video decoding unit 662.

[0161] The demultiplexing unit 651 is a processing unit similar to the demultiplexing unit 451 (FIG. 17) and performs similar processing. The control unit 653 is a processing unit similar to the control unit 453 (FIG. 17) and performs similar processing. The control information decoding unit 661 is a processing unit similar to the control information decoding unit 461 (FIG. 17) and performs similar processing. The displacement video decoding unit 662 is a processing unit similar to the displacement video decoding unit 463 (FIG. 17) and performs similar processing. The additional correction unit 654 is a processing unit similar to the additional correction unit 473 (FIG. 17) and performs similar processing.

[0162] However, the dislocation video is packed with a current displacement vector for additional correction. This current displacement vector for additional correction indicates the difference between the reference mesh and the current mesh (the difference in the positions of each vertex). The dislocation video decoding unit 662 extracts the current displacement vector for additional correction and supplies it to the additional correction unit 654. The additional correction unit 654 applies the current displacement vector for additional correction to the vertices of the supplied reference mesh. The additional correction unit 654 outputs the current mesh after the additional correction to the outside of the decoding device 650.

[0163] The present technology (method 1-2 described above) may be applied to a decoding device 650 configured as described above. For example, the displacement video decoding unit 662 of the decoding device 650 may decode encoded data and generate a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the reference mesh. Therefore, the displacement video decoding unit 662 may also be referred to as a current displacement vector decoding unit. Furthermore, the additional correction unit 654 reconstructs the current mesh by applying the current displacement vector to the vertices of the reference mesh. Therefore, the additional correction unit 654 may also be referred to as a displacement vector application unit.

[0164] With this configuration, the decoding device 650 can apply the supplied current displacement vector for additional correction to the reference mesh (the mesh to be used as the current mesh). Therefore, the decoding device 650 can suppress a decrease in encoding efficiency while suppressing a decrease in subjective quality of the mesh.

[0165] As described above, the decoding device 650 may include a control information decoding unit 661 and a control unit 653. The control information decoding unit 661 may decode the encoded data and generate control information related to the application of the current displacement vector to the vertices of the reference mesh. The control unit 653 may control the application of the current displacement vector to the vertices of the reference mesh based on the control information. With this configuration, the decoding device 650 can control the application of the current displacement vector to the vertices of the reference mesh based on the control information (i.e., according to the control of the encoding device 600). Therefore, the decoding device 650 can suppress a decrease in encoding efficiency while suppressing a decrease in the subjective quality of the mesh.

[0166] <Flow of Inter-Decoding Process> The decoding device 650 in Fig. 23 performs inter-decoding on a bitstream. An example of the flow of this inter-decoding process will be described with reference to the flowchart in Fig. 24.

[0167] The processes of steps S651 and S652 are performed in the same manner as the processes of steps S451 and S452 (FIG. 18). The process of step S653 is performed in the same manner as the process of step S456 (FIG. 18). However, in this case, the current displacement vector for additional correction indicates the difference between the reference mesh and the current mesh (the difference in the positions of each vertex).

[0168] In step S654, the additional correction unit 654 performs additional correction on the reference mesh (which is used as the current mesh) using the current displacement vector for additional correction. When the process of step S654 ends, the inter-decoding process ends.

[0169] In such an inter-decoding process, the present technology (the above-described method 1-2) may be applied. For example, in step S653, the displacement video decoding unit 662 may decode the encoded data and generate a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the reference mesh. In addition, in step S654, the additional correction unit 654 may reconstruct the current mesh by applying the current displacement vector to the vertices of the reference mesh.

[0170] By performing the inter-decoding process in this manner, the decoding device 650 can apply the supplied current displacement vector for additional correction to the reference mesh (which is used as the current mesh). Therefore, the decoding device 650 can suppress a decrease in encoding efficiency while suppressing a decrease in the subjective quality of the mesh.

[0171] 4. Reducing the amount of information in the displacement vector using up-conversion Method 2 The amount of information in the displacement vector to be transmitted may also be reduced using up-conversion. For example, as shown in the fifth row from the top of the table in Fig. 5, the current base mesh may be up-converted and the current displacement vector may be applied (Method 2).

[0172] Any method for upconverting the current base mesh may be used. For example, an upconversion filter commonly applied to meshes, such as butterfly subdivision, may be applied to the current base mesh. Such mesh upconversion filters generally improve mesh resolution and subdivide faces along the object's shape without reducing mesh quality. Thus, the current base mesh is subdivided through this upconversion to form a more accurate outline than the subdivision used for the V-DMC base mesh. Therefore, by using the difference between the upconversion result and the original mesh as the displacement vector, the amount of data required for the displacement vector can be reduced compared to when the difference between the result of the subdivision used for the V-DMC base mesh and the original mesh is used as the displacement vector. In other words, this prevents an increase in the amount of information required for the displacement vector and reduces a decrease in coding efficiency.

[0173] <Methods 2-1 and 2-2> When Method 2 is applied, the current base mesh may be up-converted in intra-encoding and intra-decoding (Method 2-1), as shown in the sixth row from the top of the table in Figure 5. Also, the current base mesh may be up-converted in inter-encoding and inter-decoding (Method 2-2), as shown in the seventh row from the top of the table in Figure 5.

[0174] As shown in Figure 25, in the case of an intraframe, the transmitted current base mesh is up-converted, and the current displacement vector is applied to the up-converted current base mesh to reconstruct the current mesh. In contrast, in the case of an interframe, the current base mesh is reconstructed using the transmitted motion vector and the reference base mesh. The reconstructed current base mesh is then up-converted. The current displacement vector is then applied to the up-converted current base mesh to reconstruct the current mesh. In this way, a reduction in coding efficiency can be suppressed in both the case of an interframe and an intraframe.

[0175] <Encoding device (intra coding)> Fig. 26 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology (the above-mentioned method 2-1) is applied. The encoding device 800 shown in Fig. 26 is a device that encodes a dynamic mesh. The encoding device 800 acquires a current mesh of the dynamic mesh, performs intra coding on it, generates a bitstream, and outputs it.

[0176] Note that Fig. 26 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in encoding device 800, there may be processing units that are not shown as blocks in Fig. 26, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 26.

[0177] 26 , the encoding device 800 includes a control unit 801, a transmission data generation unit 802, and a bitstream generation unit 803. The bitstream generation unit 803 includes a base mesh encoding unit 811, an upconverter 812, a current displacement vector derivation unit 813, a displacement video encoding unit 814, a control information encoding unit 815, and a multiplexing unit 816.

[0178] The control unit 801 is a processing unit similar to the control unit 401 (FIG. 15) and executes the same processing. The transmission data generation unit 802 is a processing unit similar to the transmission data generation unit 402 and executes the same processing.

[0179] The base mesh encoding unit 811 encodes the current base mesh supplied from the transmission data generation unit 802 and supplies the encoded data to the multiplexing unit 816 .

[0180] The up-converter 812 up-converts the current base mesh supplied from the transmission data generator 802 and supplies it to a current displacement vector deriving unit 813 .

[0181] The current displacement vector derivation unit 813 compares the upconverted current base mesh supplied from the upconverter 812 with the current mesh supplied from the transmission data generation unit 802, and generates a current displacement vector for upconversion that indicates the difference (difference in vertex positions). The current displacement vector derivation unit 813 supplies the current displacement vector for upconversion to the displacement video encoding unit 814.

[0182] The displacement video encoding unit 814 packs the current displacement vector for up-conversion into a frame of displacement video, encodes the displacement video, and provides the encoded data to the multiplexing unit 816 .

[0183] The control information encoder 815 is a processing unit similar to the control information encoder 414 and performs similar processing. The multiplexer 816 is a processing unit similar to the multiplexer 415 and performs similar processing. However, the encoded data of the displacement video includes a current displacement vector for up-conversion.

[0184] The present technology (method 2-1 described above) may be applied to the encoding device 800 configured as described above. For example, the upconverter 812 of the encoding device 800 may upconvert a current base mesh, which is a base mesh of a current frame. Furthermore, the displacement video encoding unit 814 may derive a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the upconverted current base mesh. Therefore, it may also be called a current displacement vector derivation unit.

[0185] The encoding device 800 may further include a base mesh encoding unit that encodes the current base mesh.

[0186] With this configuration, the encoding device 800 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in encoding efficiency.

[0187] <Flow of Intra-Encoding Process> The encoding device 800 in Fig. 26 performs intra-encoding on the current mesh. An example of the flow of this intra-encoding process will be described with reference to the flowchart in Fig. 27.

[0188] In step S801, the transmission data generating unit 802 generates current transmission data.

[0189] In step S802, the base mesh encoding unit 811 encodes the current base mesh.

[0190] In step S803, the upconverter 812 upconverts the current base mesh.

[0191] In step S804, the current displacement vector deriving unit 813 derives a current displacement vector for up-conversion.

[0192] In step S805, the displaced video encoding unit 814 generates a displaced video using the current displaced vector for up-conversion, and encodes the displaced video.

[0193] The processes of steps S806 to S808 are executed in the same manner as the processes of steps S406 to S407.

[0194] The present technology (method 2-1 described above) may be applied to such intra-encoding processing. For example, the upconverter 812 of the encoding device 800 may upconvert a current base mesh, which is a base mesh of the current frame, in step S803. Furthermore, the displacement video encoding unit 814 may derive a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the upconverted current base mesh in step S805. Therefore, it can also be called a current displacement vector derivation unit.

[0195] By performing the intra-coding process in this manner, the coding device 800 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in coding efficiency.

[0196] <Decoding device (intra decoding)> Fig. 28 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology (the above-mentioned method 2-1) is applied. The decoding device 850 shown in Fig. 28 is a device that intra-decodes encoded data (bit stream) to generate (reconstruct) a dynamic mesh. The decoding device 850 obtains and decodes the bit stream to generate and output a dynamic mesh.

[0197] Note that Fig. 28 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 28. In other words, in the decoding device 850, there may be processing units that are not shown as blocks in Fig. 28, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 28.

[0198] 28 , the decoding device 850 includes a demultiplexing unit 851, a decoding unit 852, a control unit 853, and a reconstruction unit 854. The decoding unit 852 includes a control information decoding unit 861, a base mesh decoding unit 862, and a displacement video decoding unit 863. The reconstruction unit 854 includes an upconverter 871 and a current mesh reconstruction unit 872.

[0199] The base mesh decoding unit 862 obtains and decodes the coded data of the base mesh extracted from the bitstream by the demultiplexing unit 851, and supplies the current base mesh to the upconverter 871. In addition, the displacement video decoding unit 863 decodes the coded data of the displacement video, extracts a current displacement vector for upconversion from the obtained displacement video, and supplies it to the current mesh reconstruction unit 872.

[0200] The upconverter 871 upconverts the supplied current base mesh and supplies it to a current mesh reconstruction unit 872. The current mesh reconstruction unit 872 applies the current displacement vector for upconversion to the upconverted current mesh, reconstructs the current mesh, and outputs it.

[0201] The present technology (method 2-1 described above) may be applied to a decoding device 850 configured as described above. For example, the upconverter 812 of the decoding device 850 may upconvert a current base mesh, which is the base mesh of the current frame. The displacement video decoding unit 863 may decode the encoded data and generate a current displacement vector indicating the difference in position between the vertices of the current mesh and the vertices of the upconverted current base mesh. Therefore, it is also referred to as a current displacement vector decoding unit. The current mesh reconstruction unit 872 may reconstruct the current mesh by applying the current displacement vector to the vertices of the upconverted current base mesh.

[0202] With this configuration, the decoding device 850 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in coding efficiency.

[0203] The decoding device 850 may further include a base mesh decoding unit that decodes the encoded data and generates a current base mesh. The upconverter 871 may upconvert the generated current base mesh.

[0204] <Flow of Intra Decoding Process> The decoding device 850 in Fig. 28 performs intra decoding on a bitstream. An example of the flow of this intra decoding process will be described with reference to the flowchart in Fig. 29.

[0205] In step S851, the demultiplexer 851 demultiplexes the bitstream to extract coded data of control information, coded data of the current base mesh, and coded data of the displacement video. In step S852, the control information decoder 861 decodes the coded data of the control information to generate (restore) the control information. In step S853, the base mesh decoder 862 decodes the coded data of the current base mesh to generate (restore) the current base mesh. In step S854, the upconverter 871 upconverts the current base mesh. In step S855, the displacement video decoder 863 decodes the coded data of the displacement video to generate (restore) the displacement video. Furthermore, the displacement video decoder 863 unpacks and derives a current displacement vector for upconversion from the displacement video. In step S856, the current mesh reconstructor 872 reconstructs the current mesh by applying the on-current displacement vector for upconversion to the upconverter. When the process of step S856 ends, the intra-decoding process ends.

[0206] The present technology (method 2-1 described above) may be applied to an intra-decoding process configured in this manner. For example, in step S854, the upconverter 812 of the decoding device 850 may upconvert a current base mesh, which is a base mesh of the current frame. In step S855, the displacement video decoding unit 863 may decode the encoded data and generate a current displacement vector indicating the difference in position between the vertices of the current mesh and the vertices of the upconverted current base mesh. Therefore, this unit is also referred to as a current displacement vector decoding unit. In step S856, the current mesh reconstruction unit 872 may reconstruct the current mesh by applying the current displacement vector to the vertices of the upconverted current base mesh.

[0207] By performing the intra-decoding process in this manner, the decoding device 850 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in coding efficiency.

[0208] <Encoding device (inter coding)> The case of inter coding is similar to the case of intra coding. Fig. 30 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology (the above-mentioned method 2-2) is applied. The encoding device 900 shown in Fig. 30 is a device that encodes a dynamic mesh. The encoding device 900 acquires a current mesh of the dynamic mesh, performs inter coding, generates a bitstream, and outputs it.

[0209] Note that Fig. 30 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in the encoding device 900, there may be processing units that are not shown as blocks in Fig. 30, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 30.

[0210] 30 , the encoding device 900 includes a control unit 901, a transmission data generation unit 902, and a bitstream generation unit 903. The bitstream generation unit 903 includes a motion vector encoding unit 911, an upconverter 912, a current displacement vector derivation unit 913, a displacement video encoding unit 914, a control information encoding unit 915, and a multiplexing unit 916.

[0211] The motion vector encoding unit 911 derives a motion vector using a current base mesh and a reference base mesh, and encodes the motion vector. The motion vector encoding unit 911 supplies encoded data of the motion vector to the multiplexing unit 916. The other processing units each perform the same processing as the processing units of the same name in the encoding device 800 (intra-encoding). For example, the upconverter 912 is a processing unit similar to the upconverter 812 ( FIG. 26 ) and performs the same processing. The current displacement vector derivation unit 913 is a processing unit similar to the current displacement vector derivation unit 813 ( FIG. 26 ) and performs the same processing. The displacement video encoding unit 914 is a processing unit similar to the displacement video encoding unit 814 ( FIG. 26 ) and performs the same processing. The control information encoding unit 915 is a processing unit similar to the control information encoding unit 815 ( FIG. 26 ) and performs the same processing. The multiplexing unit 916 is a processing unit similar to the multiplexing unit 816 ( FIG. 26 ) and performs the same processing.

[0212] The present technology (the above-described method 2-2) may be applied to an encoding device 900 configured as described above. For example, an upconverter 912 of the encoding device 900 may upconvert a current base mesh, which is a base mesh of a current frame. Furthermore, a current displacement vector derivation unit 913 of the encoding device 900 may derive a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the upconverted current base mesh. Then, a displacement video encoding unit 914 may encode the current displacement vector. Furthermore, the encoding device 900 may further include a motion vector encoding unit 911 that encodes a motion vector indicating a difference in position between the vertices of a reference base mesh, which is a base mesh of a reference frame referenced from the current frame, and the vertices of the current base mesh.

[0213] With this configuration, the encoding device 900 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in encoding efficiency.

[0214] <Flow of Inter-encoding Process> The encoding device 900 in Fig. 30 performs inter-encoding on the current mesh. An example of the flow of this inter-encoding process will be described with reference to the flowchart in Fig. 31. In this case, the processes of step S901 and step S903 to step S908 are executed in the same manner as the processes of step S801 and step S803 to step S808.

[0215] In step S902, the motion vector coding unit 911 generates and codes a motion vector. The present technology (the above-described method 2-2) may be applied to such inter-coding processing.

[0216] By performing inter-coding processing in this manner, the encoding device 900 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in encoding efficiency.

[0217] <Decoding device (inter-decoding)> Fig. 32 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which the present technology (the above-mentioned method 2-2) is applied. The decoding device 950 shown in Fig. 32 is a device that inter-decodes coded data (bit stream) to generate (reconstruct) a dynamic mesh. The decoding device 950 acquires the bit stream, decodes it, generates a dynamic mesh, and outputs it.

[0218] Note that Fig. 32 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in the decoding device 950, there may be processing units that are not shown as blocks in Fig. 32, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 32.

[0219] 32 , the decoding device 950 includes a demultiplexing unit 951, a decoding unit 952, a control unit 953, and a reconstruction unit 954. The decoding unit 952 includes a control information decoding unit 961, a motion vector decoding unit 962, and a displacement video decoding unit 963. The reconstruction unit 954 includes a current base mesh reconstruction unit 971, an upconverter 972, and a current mesh reconstruction unit 973.

[0220] The demultiplexing unit 951, the control unit 953, the control information decoding unit 961, the displacement video decoding unit 963, the upconverter 972, and the current mesh reconstruction unit 973 perform processing similar to that of the demultiplexing unit 851, the control unit 853, the control information decoding unit 861, the displacement video decoding unit 863, the upconverter 871, and the current mesh reconstruction unit 872, respectively.

[0221] The motion vector decoding unit 962 decodes the coded data of the motion vector, generates a motion vector, and supplies it to a current base mesh reconstruction unit 971. The current base mesh reconstruction unit 971 reconstructs the current base mesh by applying the motion vector to the reference base mesh. The current base mesh reconstruction unit 971 supplies the reconstructed current base mesh to an upconverter 972. The upconverter 972 upconverts the current base mesh and supplies it to a current mesh reconstruction unit 973.

[0222] The present technology (method 2-2 described above) may be applied to a decoding device 950 configured in this manner. For example, the motion vector decoding unit 962 may decode encoded data and generate motion vectors indicating the positional differences between the vertices of the reference base mesh and the vertices of the current base mesh. The current base mesh reconstruction unit 971 may reconstruct the current base mesh by applying the motion vectors to the vertices of the reference base mesh. Note that the reference base mesh is a base mesh of a reference frame referenced from the current frame, and the upconverter 972 may upconvert the reconstructed current base mesh.

[0223] With this configuration, the decoding device 950 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in coding efficiency.

[0224] <Flow of Inter-Decoding Process> An example of the flow of inter-decoding process executed by the decoding device 950 of Fig. 32 will be described with reference to the flowchart of Fig. 33. In this case, the processes of steps S951 and S952 are executed in the same manner as the processes of steps S851 and S852 of Fig. 29.

[0225] In step S953, the motion vector decoding unit 962 decodes the coded data of the motion vector to generate a motion vector.In step S954, the current base mesh reconstructing unit 971 reconstructs the current base mesh.

[0226] The processes of steps S955 to S957 are executed in the same manner as the processes of steps S854 to S857. The present technology (the above-described method 2-2) may be applied to such inter-decoding processing.

[0227] By performing the inter-decoding process in this manner, the decoding device 950 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in coding efficiency.

[0228] <5. Mode Selection> <Method 3> The above-described methods may be applied in combination with any two or more methods. Furthermore, they may be combined with methods other than those described above. When multiple methods are applicable, it may be possible to select which method to apply. That is, it may be possible to select a mode (Method 3), as shown in the bottom row of the table in FIG. 5 .

[0229] For example, the encoding device may evaluate each candidate method (mode) and select a mode to apply based on the evaluation results. In this way, a method with higher encoding efficiency can be selected. In other words, a decrease in encoding efficiency can be further suppressed.

[0230] <Encoding device> Fig. 34 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology (the above-mentioned method 3) is applied. The encoding device 1100 shown in Fig. 34 encodes a dynamic mesh. That is, the encoding device 1100 obtains a current mesh of the dynamic mesh, encodes it, generates a bitstream, and outputs it.

[0231] Note that Fig. 34 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in encoding device 1100, there may be processing units that are not shown as blocks in Fig. 34, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 34.

[0232] As shown in FIG. 34, the encoding device 1100 includes a control unit 1101 , a transmission data generation unit 1102 , an encoding unit 1103 , a decoding unit 1104 , and a mode selection unit 1105 .

[0233] The control unit 1101 controls each processing unit of the encoding device 1100. The control unit 1101 may also generate control information related to the reconstruction of the current mesh and supply it to the encoding unit 1103. The transmission data generation unit 1102 acquires a mesh and performs processing related to the generation of transmission data.

[0234] For example, the transmission data generation unit 1102 supplies information such as the current mesh to the encoding unit 1103. The transmission data generation unit 1102 also supplies the current mesh to the mode selection unit 1105.

[0235] The encoding unit 1103 encodes the supplied current mesh and supplies the encoded data (bit stream) to the decoding unit 1104 and the mode selection unit 1105. At this time, the encoding unit 1103 performs encoding using all candidate methods, including the various methods described above. The encoding unit 1103 supplies the encoded data generated by each method to the decoding unit 1104 and the mode selection unit 1105.

[0236] The decoding unit 1104 decodes the coded data (bit stream) and reconstructs the current mesh. That is, the decoding unit 1104 performs decoding using all candidate methods, including the various methods described above. The decoding unit 1104 supplies the current mesh and other data generated by each method to the mode selection unit 1105.

[0237] The mode selection unit 1105 acquires the current mesh before encoding, the encoded data of the current mesh, and the current mesh after decoding for each mode, and uses this information to calculate and compare the RDO (Rate-Distortion Optimization) costs of each mode to select the optimal mode.The mode selection unit 1105 then applies the mode selected as the optimal mode and outputs the encoded data (bitstream) generated in that mode to the outside of the encoding device 1100.This bitstream is transmitted to the decoder.The decoder then performs decoding processing in the applied mode to reconstruct the current mesh.

[0238] By applying this technology (the above-described method 3) in this way, the encoding device 1100 can suppress an increase in the amount of information of the displacement vectors to be transmitted, and suppress a decrease in encoding efficiency.

[0239] <Flow of Encoding Process> An example of the flow of the encoding process in this case will be described with reference to the flowchart in Fig. 35. When the encoding process starts, in step S1101, the transmission data generation unit 1102 generates current transmission data. In step S1102, the encoding unit 1103 performs encoding in each candidate mode. In step S1103, the decoding unit 1104 decodes the encoded data generated by encoding in each mode in that mode. In step S1104, the mode selection unit 1105 derives and compares the RDO costs for each mode, and selects a mode based on the comparison result.

[0240] The encoding process ends when the process of step S1104 ends. By performing the encoding process in this manner, the encoding device 1100 can select and apply the optimal mode from a wider variety of methods. Therefore, the encoding device 1100 can further suppress a decrease in encoding efficiency.

[0241] 6. Supplementary Notes Resolution The resolution of the original mesh after decoding may be any level. For example, the resolution of the original mesh after decoding may be substantially the same as the resolution of the original mesh before encoding. Here, "substantially the same resolution" means that the resolution is almost the same. Resolution can also be expressed as the number of vertices, connections, polygons, etc. In other words, "substantially the same resolution" means that the number of vertices, connections, polygons, etc. is approximately the same. For example, by subdividing the base mesh so as to add the number of vertices that were thinned out from the original mesh when generating the base mesh, the number of vertices of the original mesh after decoding can be made substantially the same as the number of vertices of the original mesh before encoding. In other words, the resolution of the original mesh after decoding can be made substantially the same as the resolution of the original mesh before encoding.

[0242] Furthermore, the resolution of the original mesh after decoding may be lower than the resolution of the original mesh before encoding. For example, a mesh encoding method that allows scalable decoding may be applied, and the resolution of the original mesh after decoding may be variable. For example, scalable decoding may be possible by varying the level of subdivision of the base mesh (to what level it is subdivided). In this case, for example, as described above, the resolution of the original mesh after decoding may be lower than the resolution of the original mesh before encoding.

[0243] In other words, the resolution of the reference mesh or the current mesh may be any resolution, for example, it may be substantially the same as the resolution of the original mesh before encoding, or it may be lower than the resolution of the original mesh before encoding.

[0244] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0245] FIG. 36 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0246] In a computer 1900 shown in FIG. 36, a CPU (Central Processing Unit) 1901, a ROM (Read Only Memory) 1902, and a RAM (Random Access Memory) 1903 are interconnected via a bus 1904.

[0247] An input / output interface 1910 is also connected to the bus 1904. An input unit 1911, an output unit 1912, a storage unit 1913, a communication unit 1914, and a drive 1915 are connected to the input / output interface 1910.

[0248] The input unit 1911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 1912 includes, for example, a display, a speaker, and an output terminal. The storage unit 1913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 1914 includes, for example, a network interface. The drive 1915 drives removable media 1921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0249] In a computer configured as described above, the CPU 1901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 1913 into the RAM 1903 via the input / output interface 1910 and the bus 1904 and executing the program. The RAM 1903 also stores data and the like necessary for the CPU 1901 to execute various processes.

[0250] The program executed by the computer can be applied by recording it on, for example, removable media 1921 such as package media. In this case, the program can be installed in storage unit 1913 via input / output interface 1910 by attaching removable media 1921 to drive 1915.

[0251] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 1914 and installed in the storage unit 1913.

[0252] Alternatively, this program can be installed in advance in the ROM 1902 or the storage unit 1913 .

[0253] <Application of the Present Technology> The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0254] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set in which other functions are added to a unit (e.g., a video set).

[0255] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0256] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0257] <Fields and uses to which this technology can be applied> Systems, devices, processing units, etc. to which this technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, the uses thereof are also arbitrary.

[0258] <Others> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in a bitstream, but also in a bitstream that includes differential information of the identification information relative to certain reference information. Therefore, in this specification, "flag" and "identification information" encompass not only the information itself, but also differential information relative to the reference information.

[0259] Furthermore, various types of information (e.g., metadata) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, the associated pieces of data may be combined into one piece of data or may be separate pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Furthermore, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer not to the entire data, but to only part of the data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0260] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0261] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0262] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0263] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0264] Also, for example, each step of a single flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, multiple processes included in a single step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0265] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0266] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0267] The present technology can also be configured as follows: (1) An information processing device including: a motion vector decoding unit that decodes encoded data and generates motion vectors indicating positional differences between vertices of a reference base mesh and vertices of a current base mesh; a current base mesh reconstruction unit that reconstructs the current base mesh by applying the motion vectors to the vertices of the reference base mesh; and a current mesh reconstruction unit that reconstructs the current mesh by applying a reference displacement vector to the vertices of the current base mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame, the current base mesh is a base mesh of the current frame, the base mesh is a mesh of lower resolution than an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh, the original mesh being composed of vertices and connections that represent a three-dimensional structure of the object, and the reference displacement vector indicates a positional difference between vertices obtained by subdividing the reference base mesh and the vertices of the reference mesh. (2) The information processing device according to (1), further comprising a displacement vector application unit that applies, to the vertices of the current mesh, current displacement vectors that indicate differences in position between the vertices of the current mesh and the vertices obtained by subdividing the current base mesh. (3) The information processing device according to (1) or (2), wherein the current mesh reconstruction unit applies the reference displacement vector to all of the vertices generated by subdividing the current base mesh. (4) The information processing device according to (1) or (2), wherein the current mesh reconstruction unit applies the reference displacement vector to some of the vertices generated by subdividing the current base mesh, and applies, to the remaining vertices, current displacement vectors that indicate differences in position between the vertices of the current mesh and the vertices obtained by subdividing the current base mesh.(5) The information processing device according to any one of (1) to (4), further comprising: a control information decoding unit that decodes encoded data and generates control information related to application of the reference displacement vector to vertices of the current base mesh; and a control unit that controls reconstruction of the current mesh based on the control information. (6) An information processing method that includes: decoding encoded data and generating motion vectors indicating differences in position between vertices of a reference base mesh and vertices of a current base mesh; reconstructing the current base mesh by applying the motion vectors to the vertices of the reference base mesh; reconstructing the current mesh by applying the reference displacement vector to the vertices of the current base mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame, the current base mesh is a base mesh of the current frame, the base mesh is a mesh of lower resolution than an original mesh to be coded that is composed of vertices and connections that represent a three-dimensional structure of an object, and the reference displacement vector indicates differences in position between vertices obtained by subdividing the reference base mesh and vertices of the reference mesh.

[0268] (7) An information processing device comprising: an encoding unit that encodes motion vectors indicating differences in position between vertices of a reference base mesh and vertices of a current base mesh; and a control information encoding unit that encodes control information related to application of a reference displacement vector to vertices of the current base mesh in reconstructing a current mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame, the current base mesh is a base mesh of the current frame, the base mesh is a mesh of lower resolution than an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object and is generated by thinning out the vertices from the original mesh, and the reference displacement vector indicates a difference in position between vertices obtained by subdividing the reference base mesh and vertices of the reference mesh. (8) The information processing device according to (7), further comprising: a current displacement vector derivation unit that derives current displacement vectors indicating differences in position between vertices of the current mesh and vertices of a mesh obtained by applying the reference displacement vector to the vertices of the current base mesh, and a current displacement vector encoding unit that encodes the current displacement vector. (9) The information processing device according to (7), further comprising: a current displacement vector encoding unit that encodes, for some vertices of the current mesh, a current displacement vector that indicates a difference in position between the vertices of the current base mesh and the vertices of the current mesh.(10) An information processing method comprising: encoding motion vectors indicating differences in position between vertices of a reference base mesh and vertices of a current base mesh; encoding control information regarding application of reference displacement vectors to vertices of the current base mesh in reconstructing the current mesh; the reference base mesh is a base mesh of a reference frame referenced from a current frame; the current base mesh is a base mesh of the current frame; the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh; and the reference displacement vectors indicate differences in position between vertices obtained by subdividing the reference base mesh and vertices of the reference mesh.

[0269] (11) An information processing device comprising: a current displacement vector decoding unit that decodes encoded data and generates a current displacement vector that indicates a difference in position between the vertices of a current mesh and the vertices of a reference mesh; and a displacement vector application unit that reconstructs the current mesh by applying the current displacement vector to the vertices of the reference mesh, wherein the current mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of the object in a reference frame referenced from the current frame.

[0270] (12) The information processing device according to (11), further comprising: a control information decoding unit that decodes encoded data and generates control information related to application of the current displacement vector to vertices of the reference mesh; and a control unit that controls application of the current displacement vector to the vertices of the reference mesh based on the control information. (13) An information processing method that decodes encoded data and generates current displacement vectors that indicate positional differences between vertices of a current mesh and vertices of a reference mesh, reconstructing the current mesh by applying the current displacement vectors to the vertices of the reference mesh, wherein the current mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of the object in a reference frame referenced from the current frame.

[0271] (14) An information processing device comprising: a current displacement vector derivation unit that derives a current displacement vector indicating a difference in position between a vertex of a current mesh and a vertex of a reference mesh; and a current displacement vector encoding unit that encodes the current displacement vector, wherein the current mesh is a mesh constituted by vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh constituted by vertices and connections that represent a three-dimensional structure of the object in a reference frame referenced from the current frame. (15) The information processing device according to (14), further comprising a control information encoding unit that encodes control information related to application of the current displacement vector to the vertices of the reference mesh in reconstructing the current mesh. (16) An information processing method comprising: deriving a current displacement vector indicating a difference in position between a vertex of a current mesh and a vertex of a reference mesh; encoding the current displacement vector; the current mesh being a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame; and the reference mesh being a mesh composed of vertices and connections that represent a three-dimensional structure of the object in a reference frame referenced from the current frame.

[0272] (17) An information processing device comprising: an upconversion unit that upconverts a current base mesh that is a base mesh of a current frame; a current displacement vector decoding unit that decodes encoded data and generates current displacement vectors that indicate positional differences between vertices of the current mesh and vertices of the upconverted current base mesh; and a current mesh reconstruction unit that reconstructs the current mesh by applying the current displacement vectors to the vertices of the upconverted current base mesh, wherein the base mesh is a mesh of lower resolution than an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh. (18) The information processing device according to (17), further comprising a base mesh decoding unit that decodes encoded data and generates the current base mesh, wherein the upconversion unit upconverts the generated current base mesh. (19) The information processing device according to (17), further comprising: a motion vector decoding unit that decodes encoded data and generates motion vectors indicating differences in position between vertices of a reference base mesh and vertices of the current base mesh; and a current base mesh reconstruction unit that reconstructs the current base mesh by applying the motion vectors to the vertices of the reference base mesh, wherein the reference base mesh is the base mesh of a reference frame referenced from the current frame, and the upconversion unit upconverts the reconstructed current base mesh.(20) An information processing method comprising: up-converting a current base mesh, which is a base mesh of a current frame; decoding encoded data to generate current displacement vectors indicating differences in position between vertices of the current mesh and vertices of the up-converted current base mesh; and reconstructing the current mesh by applying the current displacement vectors to the vertices of the up-converted current base mesh, wherein the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh.

[0273] (21) An information processing device comprising: an upconversion unit that upconverts a current base mesh that is a base mesh of a current frame; a current displacement vector derivation unit that derives current displacement vectors that indicate positional differences between vertices of the current mesh and vertices of the upconverted current base mesh; and a current displacement vector encoding unit that encodes the current displacement vector, wherein the base mesh is a mesh of lower resolution than an original mesh to be encoded that is composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh. (22) The information processing device according to (21), further comprising a base mesh encoding unit that encodes the current base mesh. (23) The information processing device according to (21), further comprising a motion vector encoding unit that encodes motion vectors that indicate positional differences between vertices of a reference base mesh that is the base mesh of a reference frame referenced from the current frame and the vertices of the current base mesh. (24) An information processing method including: up-converting a current base mesh, which is a base mesh of a current frame; deriving a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the up-converted current base mesh; and encoding the current displacement vector, wherein the base mesh is a mesh of lower resolution than the original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh.

[0274] 200 Encoding device, 201 Control unit, 202 Transmission data generation unit, 203 Bit stream generation unit, 211 Motion vector encoding unit, 212 Displacement video encoding unit, 213 Control information encoding unit, 214 Multiplexing unit, 250 Decoding device, 251 Demultiplexing unit, 252 Decoding unit, 253 Control unit, 254 Reconstruction unit, 261 Control information decoding unit, 262 Motion vector decoding unit, 263 Displacement video decoding unit, 271 Current base mesh reconstruction unit, 272 Current mesh reconstruction unit, 1900 Computer

Claims

1. An information processing device comprising: a motion vector decoding unit that decodes encoded data and generates motion vectors indicating differences in position between vertices of a reference base mesh and vertices of a current base mesh; a current base mesh reconstruction unit that reconstructs the current base mesh by applying the motion vectors to the vertices of the reference base mesh; and a current mesh reconstruction unit that reconstructs the current mesh by applying a reference displacement vector to the vertices of the current base mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame, the current base mesh is a base mesh of the current frame, the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh, the original mesh being generated by thinning out the vertices from the original mesh, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and the reference displacement vector indicates differences in position between vertices obtained by subdividing the reference base mesh and the vertices of the reference mesh.

2. The information processing device according to claim 1, further comprising a displacement vector application unit that applies, to the vertices of the current mesh, a current displacement vector indicating a difference in position between a vertex obtained by subdividing the current base mesh and the vertex of the current mesh.

3. The information processing device according to claim 1, wherein the current mesh reconstruction unit applies the reference displacement vector to all vertices generated by subdivision of the current base mesh.

4. The information processing device according to claim 1, wherein the current mesh reconstruction unit applies the reference displacement vector to a portion of the vertices generated by subdivision of the current base mesh, and applies a current displacement vector indicating a difference in position between the vertices obtained by subdivision of the current base mesh and the vertices of the current mesh to the remaining vertices.

5. The information processing device according to claim 1, further comprising: a control information decoding unit that decodes encoded data and generates control information regarding application of the reference displacement vector to vertices of the current base mesh; and a control unit that controls reconstruction of the current mesh based on the control information.

6. An information processing method comprising: decoding encoded data, generating motion vectors indicating differences in position between vertices of a reference base mesh and vertices of a current base mesh; reconstructing the current base mesh by applying the motion vectors to the vertices of the reference base mesh; reconstructing a current mesh by applying a reference displacement vector to the vertices of the current base mesh; the reference base mesh is a base mesh of a reference frame referenced from a current frame; the current base mesh is a base mesh of the current frame; the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh; and the reference displacement vector indicates differences in position between vertices obtained by subdividing the reference base mesh and the vertices of the reference mesh.

7. An information processing device comprising: an encoding unit that encodes motion vectors indicating differences in position between vertices of a reference base mesh and vertices of a current base mesh; and a control information encoding unit that encodes control information regarding application of reference displacement vectors to vertices of the current base mesh in reconstructing the current mesh, wherein the reference base mesh is a base mesh of a reference frame referenced from a current frame; the current base mesh is a base mesh of the current frame; the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh; and the reference displacement vector indicates differences in position between vertices obtained by subdividing the reference base mesh and the vertices of the reference mesh.

8. The information processing device according to claim 7, further comprising: a current displacement vector derivation unit that derives a current displacement vector indicating a difference in position between a vertex of the current mesh and a vertex of a mesh to which the reference displacement vector has been applied to the vertices of the current base mesh; and a current displacement vector encoding unit that encodes the current displacement vector.

9. The information processing device according to claim 7, further comprising a current displacement vector encoding unit that encodes a current displacement vector indicating a difference in position between some vertices of the current mesh and vertices of the current base mesh.

10. An information processing method comprising: encoding motion vectors indicating differences in position between vertices of a reference base mesh and vertices of a current base mesh; encoding control information regarding application of reference displacement vectors to vertices of the current base mesh in reconstructing the current mesh; the reference base mesh is a base mesh of a reference frame referenced from a current frame; the current base mesh is a base mesh of the current frame; the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh; and the reference displacement vectors indicate differences in position between vertices obtained by subdividing the reference base mesh and the vertices of the reference mesh.

11. An information processing device comprising: a current displacement vector decoding unit that decodes encoded data and generates a current displacement vector indicating a difference in position between the vertices of a current mesh and the vertices of a reference mesh; and a displacement vector application unit that reconstructs the current mesh by applying the current displacement vector to the vertices of the reference mesh, wherein the current mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh composed of vertices and connections that represent the three-dimensional structure of the object in a reference frame referenced from the current frame.

12. An information processing method comprising: decoding encoded data, generating current displacement vectors indicating differences in position between vertices of a current mesh and vertices of a reference mesh; reconstructing the current mesh by applying the current displacement vectors to the vertices of the reference mesh; the current mesh being a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame; and the reference mesh being a mesh composed of vertices and connections that represent the three-dimensional structure of the object in a reference frame referenced from the current frame.

13. An information processing device comprising: a current displacement vector derivation unit that derives a current displacement vector indicating a difference in position between vertices of a current mesh and vertices of a reference mesh; and a current displacement vector encoding unit that encodes the current displacement vector, wherein the current mesh is a mesh composed of vertices and connections that represent a three-dimensional structure of an object in a current frame, and the reference mesh is a mesh composed of vertices and connections that represent the three-dimensional structure of the object in a reference frame referenced from the current frame.

14. An information processing method comprising: deriving a current displacement vector indicating a difference in position between vertices of a current mesh and vertices of a reference mesh; encoding the current displacement vector; the current mesh being a mesh composed of vertices and connections representing a three-dimensional structure of an object in a current frame; and the reference mesh being a mesh composed of vertices and connections representing the three-dimensional structure of the object in a reference frame referenced from the current frame.

15. An information processing device comprising: an up-conversion unit that up-converts a current base mesh, which is a base mesh of a current frame; a current displacement vector decoding unit that decodes encoded data and generates a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the up-converted current base mesh; and a current mesh reconstruction unit that reconstructs the current mesh by applying the current displacement vector to the vertices of the up-converted current base mesh, wherein the base mesh is a mesh of lower resolution than that of an original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh.

16. The information processing device according to claim 15, further comprising a base mesh decoding unit that decodes encoded data and generates the current base mesh, wherein the up-conversion unit up-converts the generated current base mesh.

17. An information processing device as described in claim 15, further comprising: a motion vector decoding unit that decodes encoded data and generates motion vectors indicating the positional differences between vertices of a reference base mesh and the vertices of the current base mesh; and a current base mesh reconstruction unit that reconstructs the current base mesh by applying the motion vectors to the vertices of the reference base mesh, wherein the reference base mesh is the base mesh of a reference frame referenced from the current frame, and the up-conversion unit up-converts the reconstructed current base mesh.

18. An information processing method comprising: up-converting a current base mesh, which is a base mesh of a current frame; decoding encoded data to generate a current displacement vector indicating a difference in position between vertices of the current mesh and vertices of the up-converted current base mesh; and reconstructing the current mesh by applying the current displacement vector to the vertices of the up-converted current base mesh, wherein the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh.

19. An information processing device comprising: an up-conversion unit that up-converts a current base mesh, which is a base mesh of a current frame; a current displacement vector derivation unit that derives a current displacement vector indicating a difference in position between the vertices of the current mesh and the vertices of the up-converted current base mesh; and a current displacement vector encoding unit that encodes the current displacement vector, wherein the base mesh is a mesh of lower resolution than that of an original mesh to be encoded, which is composed of vertices and connections that represent the three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh to be encoded.

20. An information processing method comprising: up-converting a current base mesh, which is a base mesh of a current frame; deriving a current displacement vector indicating a difference in position between vertices of the current mesh and vertices of the up-converted current base mesh; and encoding the current displacement vector; wherein the base mesh is a mesh of lower resolution than an original mesh to be encoded, the original mesh being composed of vertices and connections that represent a three-dimensional structure of an object, and is generated by thinning out the vertices from the original mesh.

Images