Encoding method, decoding method, encoding device, and decoding device

Abstract

NOT VISIBLE DUE TO STATUS OF PATENT

Description

Encoding method, decoding method, encoding device, and decoding device

[0001] The present disclosure relates to encoding methods and the like.

[0002] In US Pat. No. 6,299,549 a method and apparatus for encoding and decoding three-dimensional mesh data is proposed.

[0003] Japanese Patent Application Laid-Open No. 2006-187015

[0004] Further improvements are desired in the process of encoding or decoding the coordinates of the vertices of a three-dimensional mesh.The present disclosure aims to improve the process of encoding or decoding the coordinates of the vertices of a three-dimensional mesh.

[0005] An encoding method according to one aspect of the present invention is an encoding method for encoding the coordinates of vertices included in a three-dimensional mesh, which derives a predicted value of the coordinates of the first vertex using the coordinates of three vertices different from the first vertex out of a plurality of vertices connected to a second vertex connected to a first vertex to be encoded, and encodes the difference value between the predicted value and the coordinates of the first vertex.

[0006] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, and a recording medium.

[0007] The present disclosure may contribute to improvements in encoding processes related to the coordinates of vertices of a three-dimensional mesh.

[0008] 1 is a conceptual diagram showing a three-dimensional mesh according to an embodiment. FIG. 2 is a conceptual diagram showing basic elements of a three-dimensional mesh according to an embodiment. FIG. 3 is a conceptual diagram showing mapping according to an embodiment. FIG. 4 is a block diagram showing a configuration example of an encoding / decoding system according to an embodiment. FIG. 5 is a block diagram showing a configuration example of an encoding device according to an embodiment. FIG. 6 is a block diagram showing another configuration example of an encoding device according to an embodiment. FIG. 7 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 8 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 9 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 10 is a conceptual diagram showing another configuration example of a bit stream according to an embodiment. FIG. 11 is a conceptual diagram showing yet another configuration example of a bit stream according to an embodiment. FIG. 12 is a block diagram showing a specific example of an encoding / decoding system according to an embodiment. FIG. 13 is a conceptual diagram showing an example configuration of point cloud data according to an embodiment. FIG. 14 is a conceptual diagram showing an example data file of point cloud data according to an embodiment. FIG. 15 is a conceptual diagram showing an example configuration of mesh data according to an embodiment. FIG. 16 is a conceptual diagram showing an example data file of mesh data according to an embodiment. FIG. 17 is a conceptual diagram showing types of three-dimensional data according to an embodiment. FIG. 18 is a block diagram showing an example configuration of a three-dimensional data encoder according to an embodiment. FIG. 19 is a block diagram showing an example configuration of a three-dimensional data decoder according to an embodiment. FIG. 19 is a block diagram showing another configuration example of a three-dimensional data encoder according to an embodiment. FIG. 19 is a block diagram showing another configuration example of a three-dimensional data decoder according to an embodiment. FIG. 1 is a conceptual diagram showing a specific example of encoding processing according to an embodiment. FIG. 2 is a conceptual diagram showing a specific example of decoding processing according to an embodiment. FIG. 3 is a block diagram showing an implementation example of an encoding device according to an embodiment. FIG. 4 is a block diagram showing an implementation example of a decoding device according to an embodiment. FIG. 5 is a block diagram showing another configuration example of an encoding / decoding system according to an embodiment. FIG. 6 is a block diagram showing another configuration example of an encoding device according to an embodiment. FIG. 7 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 8 is a flow diagram showing processing by an encoding device according to an embodiment. FIG. 9 is an explanatory diagram conceptually showing encoding of a mesh frame according to an embodiment. FIG. 10 is a flow diagram showing processing by a decoding device according to an embodiment. FIG. 11 is an explanatory diagram conceptually showing decoding of a mesh frame according to an embodiment.FIG. 1 is an explanatory diagram showing an example of a method for predicting vertex coordinates according to an embodiment. FIG. 2 is an explanatory diagram showing an example of a method for predicting vertex coordinates according to an embodiment. FIG. 3 is an explanatory diagram showing an example of a method for predicting vertex coordinates according to an embodiment. FIG. 4 is an explanatory diagram showing an example of a method for predicting vertex coordinates according to an embodiment. FIG. 5 is an explanatory diagram showing an example of a method for predicting vertex coordinates according to an embodiment. FIG. 6 is a flow diagram showing an example of processing by an encoding device according to an embodiment. FIG. 7 is a flow diagram showing an example of processing by a decoding device according to an embodiment. FIG. 8 is a flow diagram showing an example of processing by an encoding device according to an embodiment. FIG. 9 is a flow diagram showing an example of processing by a decoding device according to an embodiment. FIG. 10 is an explanatory diagram showing an example of a method for predicting vertex coordinates according to an embodiment. FIG. 11 is an explanatory diagram showing an example of a method for predicting vertex coordinates according to an embodiment. FIG. 12 is a flow diagram showing an example of a process for determining predicted values ​​according to an embodiment. FIG. 13 is a flow diagram showing an example of encoding processing according to an embodiment. FIG. 14 is a flow diagram showing an example of decoding processing according to an embodiment.

[0009] <Summary of the Disclosure> Three-dimensional (3D) meshes are used in computer graphics images, for example, which may be composed of multiple temporally distinct frames, each of which may be represented by a 3D mesh.

[0010] A 3D mesh is composed of vertex information indicating the positions of each of the vertices in 3D space, connectivity information indicating the connections between the vertices, and attribute information indicating the attributes of each vertex or face. Each face is constructed according to the connectivity between the vertices. Various computer graphics images can be expressed using such 3D meshes.

[0011] Furthermore, for transmission and storage of the 3D mesh, efficient encoding and decoding of the 3D mesh is expected. For efficient encoding and decoding of the 3D mesh, arithmetic coding and decoding may be used.

[0012] Further improvements are desired in the encoding or decoding process for three-dimensional data. The present disclosure aims to improve the encoding or decoding process for three-dimensional data.

[0013] Below, examples of inventions that can be obtained from the disclosure of this specification will be given, and the effects and the like that can be obtained from these inventions will be explained.

[0014] (1) An encoding method for encoding the coordinates of vertices included in a three-dimensional mesh, which derives a predicted value of the coordinates of the first vertex using the coordinates of three vertices different from the first vertex out of multiple vertices connected to a second vertex connected to a first vertex to be encoded, and encodes the difference value between the predicted value and the coordinates of the first vertex.

[0015] According to the above aspect, a predicted value of the coordinates of the first vertex is derived using the coordinates of three vertices other than the first vertex among a plurality of vertices connected to a second vertex connected to the first vertex to be encoded. When the positional correlation between the coordinates of the three vertices and the first vertex is relatively strong, the derived predicted value may be an appropriate value that is relatively close to the coordinates of the three vertices. This may improve the accuracy of encoding the first vertex and may reduce the amount of code. In this way, the above encoding method may contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh.

[0016] (2) The encoding method according to (1), wherein the predicted value is derived without using the coordinates of the second vertex.

[0017] According to the above aspect, when deriving a predicted value, the coordinates of the three vertices are used, in other words, the "second vertex connected to the first vertex to be coded" is not used, which may make it possible to more easily derive an appropriate predicted value. In this way, the above coding method can more easily contribute to improving the coding process for the coordinates of the vertices of a 3D mesh.

[0018] (3) The encoding method described in (1), in which, when deriving the predicted value, the predicted value is derived using three vertices, including at least a third vertex and a fourth vertex, which are two vertices connected to the first vertex.

[0019] According to the above aspect, when deriving a predicted value, by using the coordinates of at least three vertices including the third vertex and the fourth vertex, which are two vertices connected to the first vertex, it is possible to more easily derive an appropriate predicted value. In this way, the above encoding method can more easily contribute to improving the encoding process for the coordinates of the vertices of a 3D mesh.

[0020] (4) The encoding method described in (1), in which, when deriving the predicted value, the predicted value is derived using three vertices including at least a fifth vertex that is not connected to the first vertex.

[0021] According to the above aspect, by using the coordinates of at least three vertices, including the fifth vertex that is not connected to the first vertex, when deriving a predicted value, it is possible to more easily derive an appropriate predicted value. In this way, the above encoding method can more easily contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh.

[0022] (5) The encoding method described in (3), wherein when deriving the predicted value, the predicted value is derived using three vertices including a fifth vertex that is not connected to the first vertex, and the difference value between the coordinates of the fourth vertex and the coordinates of the fifth vertex is twice the difference value between the predicted value of the coordinates of the first vertex and the coordinates of the third vertex.

[0023] According to the above aspect, when deriving a predicted value, by calculating the coordinates of one vertex out of the four vertices of a trapezoid formed by three vertices, including the third vertex and the fourth vertex, which are two vertices connected to the first vertex, and the fifth vertex, which is a vertex not connected to the first vertex, the encoding method can more easily contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh.

[0024] (6) The encoding method according to (1), wherein the predicted value is a first predicted value, further comprising: deriving a second predicted value that is a predicted value of the coordinates of the first vertex using coordinates of three new vertices that are different from the first vertex, among a plurality of new vertices that are connected to a new second vertex that is connected to the first vertex and that are at least partially different from the plurality of vertices; deriving a third predicted value that is a new predicted value of the coordinates of the first vertex using the first predicted value and the second predicted value; and encoding a new difference value between the third predicted value and the coordinates of the first vertex.

[0025] According to the above aspect, a new third predicted value is derived using a second predicted value derived using three new vertices that are at least partially different from the three vertices. The derived third predicted value may be a more appropriate predicted value than the first predicted value or the second predicted value. This may improve the accuracy of encoding the first vertex and may reduce the amount of code. In this way, the above encoding method may contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh.

[0026] (7) The encoding method according to (6), wherein, when deriving the third predicted value, an average of the first predicted value and the second predicted value is derived as the third predicted value.

[0027] According to the above aspect, the third predicted value can be calculated more easily by performing information processing to take the average of the first predicted value and the second predicted value. Therefore, the above encoding method can contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh by performing easier information processing.

[0028] (8) The encoding method according to (6), wherein when deriving the third predicted value, a weighted average of the first predicted value and the second predicted value is derived as the third predicted value.

[0029] According to the above aspect, the third predicted value can be calculated more easily by information processing that takes a weighted average of the first predicted value and the second predicted value. Therefore, the above encoding method can contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh by easier information processing.

[0030] (9) The encoding method described in (1) further acquires connection information indicating the connections of the vertices included in the three-dimensional mesh, and when deriving the predicted value, identifies the three vertices using the connections between the vertices included in the three-dimensional mesh indicated in the acquired connection information, and derives the predicted value of the coordinates of the first vertex using the coordinates of the identified three vertices.

[0031] According to the above aspect, the three vertices can be more easily identified by using the connectivity information, and the predicted value can be derived using the identified three vertices. Therefore, the above encoding method can more easily contribute to improving the encoding process for the coordinates of the vertices of a 3D mesh.

[0032] (10) A decoding method for decoding the coordinates of vertices included in a three-dimensional mesh, the decoding method comprising: deriving a predicted value of the coordinates of the first vertex using the coordinates of three vertices different from the first vertex among a plurality of vertices connected to a second vertex connected to a first vertex to be decoded; decoding a difference value between the predicted value and the coordinate of the first vertex; and restoring the coordinate of the first vertex using the predicted value and the difference value.

[0033] According to the above aspect, a predicted value of the coordinates of the first vertex is derived using the coordinates of three vertices different from the first vertex among a plurality of vertices connected to a second vertex connected to the first vertex to be decoded. When the positional correlation between the coordinates of the three vertices and the first vertex is relatively strong, the derived predicted value may be an appropriate value that is relatively close to the coordinates of the three vertices. This may improve the accuracy of decoding the first vertex and may reduce the amount of code. In this way, the above decoding method may contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh.

[0034] (11) The decoding method according to (10), wherein the predicted value is derived without using the coordinates of the second vertex.

[0035] According to the above aspect, when deriving a predicted value, the coordinates of the three vertices are used, in other words, the "second vertex connected to the first vertex to be decoded" is not used, which may make it possible to more easily derive an appropriate predicted value. In this way, the above decoding method can more easily contribute to improving the decoding process for the coordinates of the vertices of a 3D mesh.

[0036] (12) A decoding method according to (10), in which, when deriving the predicted value, the predicted value is derived using three vertices, including at least a third vertex and a fourth vertex, which are two vertices connected to the first vertex.

[0037] According to the above aspect, when deriving a predicted value, by using the coordinates of at least three vertices including the third vertex and the fourth vertex, which are two vertices connected to the first vertex, it is possible to more easily derive an appropriate predicted value. In this way, the above decoding method can more easily contribute to improving the decoding process for the coordinates of the vertices of a 3D mesh.

[0038] (13) A decoding method according to (10), in which, when deriving the predicted value, the predicted value is derived using three vertices including at least a fifth vertex that is not connected to the first vertex.

[0039] According to the above aspect, when deriving a predicted value, by using the coordinates of at least three vertices including the first vertex and the fifth vertex that is not connected to the first vertex, it is possible to more easily derive an appropriate predicted value. In this way, the above decoding method can more easily contribute to improving the decoding process for the coordinates of the vertices of a 3D mesh.

[0040] (14) When deriving the predicted value, the three vertices further include a fifth vertex that is not connected to the first vertex, and the difference value between the coordinates of the fourth vertex and the coordinates of the fifth vertex is twice the difference value between the predicted value of the coordinates of the first vertex and the coordinates of the third vertex. This is a decoding method described in (12).

[0041] According to the above aspect, when deriving a predicted value, by calculating the coordinates of one vertex out of four vertices of a trapezoid formed by three vertices, including the third vertex and the fourth vertex, which are two vertices connected to the first vertex, and the fifth vertex, which is a vertex not connected to the first vertex, the trapezoid may be more easily derived with an appropriate predicted value. In this way, the above decoding method may contribute to more easily improving the decoding process for the coordinates of the vertices of a three-dimensional mesh.

[0042] (15) The decoding method according to (10), wherein the predicted value is a first predicted value, further comprising: deriving a second predicted value that is a predicted value of the coordinates of the first vertex using coordinates of three new vertices that are different from the first vertex, among a plurality of new vertices that are connected to a new second vertex that is connected to the first vertex and that are at least partially different from the plurality of vertices; deriving a third predicted value that is a new predicted value of the coordinates of the first vertex using the first predicted value and the second predicted value; decoding a new difference value between the third predicted value and the coordinates of the first vertex; and restoring the coordinates of the first vertex using the third predicted value and the new difference value.

[0043] According to the above aspect, a new third predicted value is derived using a second predicted value derived using three new vertices that are at least partially different from the three vertices. The derived third predicted value may be a more appropriate predicted value than the first predicted value or the second predicted value. This may improve the accuracy of decoding the first vertex and may reduce the amount of code. In this way, the above decoding method may contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh.

[0044] (16) The decoding method according to (15), wherein when deriving the third predicted value, an average of the first predicted value and the second predicted value is derived as the third predicted value.

[0045] According to the above aspect, the third predicted value can be calculated more easily by performing information processing to take the average of the first predicted value and the second predicted value. Therefore, the above decoding method can contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh by performing easier information processing.

[0046] (17) The decoding method according to (15), wherein when deriving the third predicted value, a weighted average of the first predicted value and the second predicted value is derived as the third predicted value.

[0047] According to the above aspect, the third predicted value can be calculated more easily by performing information processing to take a weighted average of the first predicted value and the second predicted value. Therefore, the above decoding method can contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh by performing easier information processing.

[0048] (18) The decoding method described in (10) further acquires connection information indicating the connections of the vertices included in the three-dimensional mesh, and when deriving the predicted value, identifies the three vertices using the connections between the vertices included in the three-dimensional mesh indicated in the acquired connection information, and derives the predicted value of the coordinates of the first vertex using the coordinates of the identified three vertices.

[0049] According to the above aspect, the three vertices can be more easily identified by using the connection information, and the predicted value can be derived using the identified three vertices. Therefore, the above decoding method can more easily contribute to improving the decoding process for the coordinates of the vertices of the 3D mesh.

[0050] (19) An encoding device comprising a memory and a circuit that can access the memory, wherein the circuit executes an encoding method in operation to encode the coordinates of vertices included in a three-dimensional mesh, and the encoding method derives a predicted value of the coordinates of the first vertex using the coordinates of three vertices different from the first vertex out of multiple vertices that are connected to a first vertex to be encoded and a second vertex that is connected to the first vertex, and encodes the difference value between the predicted value and the coordinates of the first vertex.

[0051] According to the above aspect, the encoding device achieves the same effects as the above encoding method.

[0052] (20) A decoding device comprising a memory and a circuit that can access the memory, wherein the circuit executes a decoding method in operation for decoding coordinates of vertices included in a three-dimensional mesh, and the decoding method derives a predicted value of the coordinates of the first vertex using coordinates of three vertices different from the first vertex out of a plurality of vertices that are connected to a second vertex that is connected to a first vertex to be decoded, decodes a difference value between the predicted value and the coordinate of the first vertex, and restores the coordinate of the first vertex using the predicted value and the difference value.

[0053] According to the above aspect, the decoding device achieves the same effects as the above decoding method.

[0054] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, or a recording medium.

[0055] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0056] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

[0057] (Embodiment) In this embodiment, an encoding method, a decoding method, etc. will be described.

[0058] <Expressions and Terms> The following expressions and terms are used herein.

[0059] (1) Three-dimensional Mesh A three-dimensional mesh is a collection of multiple faces, and represents, for example, a three-dimensional object. A three-dimensional mesh is mainly composed of vertex information, connectivity information, and attribute information. A three-dimensional mesh may be expressed as a polygon mesh or a mesh. A three-dimensional mesh may also vary over time. A three-dimensional mesh may include metadata related to the vertex information, connectivity information, and attribute information, and may also include other additional information.

[0060] (2) Vertex Information Vertex information is information indicating a vertex. For example, the vertex information indicates the position of a vertex in a three-dimensional space. Furthermore, a vertex corresponds to a vertex of a face that constitutes a three-dimensional mesh. Vertex information may be expressed as "geometry." Furthermore, vertex information may be expressed as position information.

[0061] (3) Connection Information Connection information is information that indicates connections between vertices. For example, connection information indicates connections for forming faces or edges of a three-dimensional mesh. Connection information may be expressed as "Connectivity." Connection information may also be expressed as face information.

[0062] (4) Attribute Information Attribute information is information that indicates attributes of a vertex or a face. For example, attribute information indicates attributes such as a color, an image, and a normal vector associated with a vertex or a face. Attribute information may be expressed as "texture."

[0063] (5) Faces A face is an element that constitutes a three-dimensional mesh. Specifically, a face is a polygon on a plane in three-dimensional space. For example, a face can be defined as a triangle in three-dimensional space.

[0064] (6) Plane A plane is a two-dimensional plane in a three-dimensional space. For example, a polygon is formed on a plane, and multiple polygons are formed on multiple planes.

[0065] (7) Bitstream: A bitstream corresponds to coded information. A bitstream may also be referred to as a stream, a coded bitstream, a compressed bitstream, or a coded signal.

[0066] (8) Encoding and Decoding The term encoding may be substituted with terms such as storing, including, writing, describing, signaling, sending, notifying, saving, or compressing, and these terms may be interchangeable. For example, encoding information may mean including the information in a bitstream. Also, encoding information into a bitstream may mean encoding the information to generate a bitstream that includes the encoded information.

[0067] Additionally, the term "decode" may be replaced with terms such as "read," "decode," "read," "load," "derive," "obtain," "receive," "extract," "reconstruct," "reconstruct," "decompress," or "decompress," and these terms may be interchangeable. For example, decoding information may mean obtaining information from a bitstream. Decoding information from a bitstream may mean decoding the bitstream to obtain information contained in the bitstream.

[0068] (9) Ordinal Numbers In the description, ordinal numbers such as first and second may be assigned to components, etc. These ordinal numbers may be changed as appropriate. Furthermore, new ordinal numbers may be assigned to components, etc., or removed. Furthermore, these ordinal numbers may be assigned to elements in order to identify them, and may not correspond to a meaningful order.

[0069] <Three-dimensional mesh> Fig. 1 is a conceptual diagram showing a three-dimensional mesh according to this embodiment. A three-dimensional mesh is composed of multiple faces. For example, each face is a triangle. The vertices of these triangles are defined in three-dimensional space. The three-dimensional mesh then represents a three-dimensional object. Each face may have a color or an image.

[0070] FIG. 2 is a conceptual diagram showing the basic elements of a three-dimensional mesh according to this embodiment. A three-dimensional mesh is composed of vertex information, connection information, and attribute information. The vertex information indicates the positions of the vertices of a face in three-dimensional space. The connection information indicates the connections between the vertices. A face can be identified by the vertex information and connection information. In other words, a colorless three-dimensional object is formed in three-dimensional space by the vertex information and connection information.

[0071] The attribute information may be associated with a vertex or a face. The attribute information associated with a vertex may be expressed as "Attribute Per Point." The attribute information associated with a vertex may indicate an attribute of the vertex itself, or may indicate an attribute of a face connected to the vertex.

[0072] For example, a color may be associated with a vertex as attribute information. The color associated with a vertex may be the color of the vertex itself, or the color of a face connected to the vertex. The color of a face may be the average of multiple colors associated with multiple vertices of the face. Furthermore, a normal vector may be associated with a vertex or a face as attribute information. Such a normal vector can represent the front and back of a face.

[0073] A two-dimensional image may be associated with a surface as attribute information. The two-dimensional image associated with a surface is also expressed as a texture image or an "Attribute Map." Information indicating mapping between the surface and the two-dimensional image may be associated with the surface as attribute information. Information indicating such mapping may be expressed as mapping information, vertex information of a texture image, texture coordinates, or "Attribute UV Coordinate."

[0074] Furthermore, information such as color, image, and moving image used as attribute information may be expressed as "parametric space."

[0075] The attribute information allows texture to be reflected on the three-dimensional object. That is, a three-dimensional object having color is formed in three-dimensional space based on the vertex information, connection information, and attribute information.

[0076] In the above, the attribute information is associated with the vertices or faces, but it may also be associated with the edges.

[0077] 3 is a conceptual diagram illustrating mapping according to this embodiment. For example, a region of a two-dimensional image on a two-dimensional plane can be mapped onto a surface of a three-dimensional mesh in three-dimensional space. Specifically, coordinate information of the region in the two-dimensional image is associated with the surface of the three-dimensional mesh. As a result, an image of the mapped region in the two-dimensional image is reflected on the surface of the three-dimensional mesh.

[0078] By using the mapping, the 2D image used as attribute information can be separated from the 3D mesh. For example, in encoding the 3D mesh, the 2D image may be encoded by an image encoding method or a video encoding method.

[0079] <System Configuration> Fig. 4 is a block diagram showing an example of the configuration of a coding / decoding system according to this embodiment. In Fig. 4, the coding / decoding system includes a coding device 100 and a decoding device 200.

[0080] For example, the encoding device 100 obtains a three-dimensional mesh and encodes the three-dimensional mesh into a bitstream. Then, the encoding device 100 outputs the bitstream to the network 300. For example, the bitstream includes the encoded three-dimensional mesh and control information for decoding the encoded three-dimensional mesh. By encoding the three-dimensional mesh, information about the three-dimensional mesh is compressed.

[0081] The network 300 transmits a bitstream from the encoding device 100 to the decoding device 200. The network 300 may be the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 300 is not necessarily limited to bidirectional communication, and may be a unidirectional communication network for terrestrial digital broadcasting, satellite broadcasting, or the like.

[0082] Furthermore, the network 300 can be replaced by a recording medium such as a DVD (Digital Versatile Disc) or a BD (Blu-Ray Disc (registered trademark)).

[0083] The decoding device 200 obtains a bitstream and decodes a three-dimensional mesh from the bitstream. By decoding the three-dimensional mesh, information about the three-dimensional mesh is expanded. For example, the decoding device 200 decodes the three-dimensional mesh according to a decoding method corresponding to the encoding method used by the encoding device 100 to encode the three-dimensional mesh. That is, the encoding device 100 and the decoding device 200 perform encoding and decoding according to encoding methods and decoding methods that correspond to each other.

[0084] The 3D mesh before encoding may also be referred to as an original 3D mesh, and the 3D mesh after decoding may also be referred to as a reconstructed 3D mesh.

[0085] 5 is a block diagram showing an example of the configuration of a coding device 100 according to this embodiment. For example, the coding device 100 includes a vertex information encoder 101, a connection information encoder 102, and an attribute information encoder 103.

[0086] The vertex information encoder 101 is an electrical circuit that encodes vertex information. For example, the vertex information encoder 101 encodes the vertex information into a bitstream according to a format defined for the vertex information.

[0087] The connection information encoder 102 is an electrical circuit that encodes the connection information, for example, the connection information encoder 102 encodes the connection information into a bitstream according to a format defined for the connection information.

[0088] The attribute information encoder 103 is an electric circuit that encodes the attribute information. For example, the attribute information encoder 103 encodes the attribute information into a bit stream in accordance with a format defined for the attribute information.

[0089] The vertex information, connectivity information, and attribute information may be coded using variable-length coding or fixed-length coding, such as Huffman coding or context-adaptive binary arithmetic coding (CABAC).

[0090] The vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be integrated together, or each of the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be further subdivided into multiple components.

[0091] 6 is a block diagram showing another example of the configuration of the encoding device 100 according to this embodiment. For example, the encoding device 100 includes a pre-processor 104 and a post-processor 105 in addition to the configuration shown in FIG.

[0092] The preprocessor 104 is an electrical circuit that performs processing before encoding the vertex information, connectivity information, and attribute information. For example, the preprocessor 104 may perform a conversion process, a separation process, a multiplexing process, or the like on the 3D mesh before encoding. More specifically, for example, the preprocessor 104 may separate the vertex information, connectivity information, and attribute information from the 3D mesh before encoding.

[0093] The post-processor 105 is an electrical circuit that performs processing after the vertex information, connection information, and attribute information are encoded. For example, the post-processor 105 may perform conversion processing, separation processing, multiplexing processing, or the like on the encoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 105 may multiplex the encoded vertex information, connection information, and attribute information into a bitstream. Furthermore, for example, the post-processor 105 may further perform variable-length coding on the encoded vertex information, connection information, and attribute information.

[0094] 7 is a block diagram showing an example of the configuration of a decoding device 200 according to this embodiment. For example, the decoding device 200 includes a vertex information decoder 201, a connection information decoder 202, and an attribute information decoder 203.

[0095] The vertex information decoder 201 is an electrical circuit that decodes vertex information. For example, the vertex information decoder 201 decodes vertex information from a bitstream according to a format defined for the vertex information.

[0096] The connection information decoder 202 is an electrical circuit that decodes the connection information, for example, the connection information decoder 202 decodes the connection information from the bitstream according to a format defined for the connection information.

[0097] The attribute information decoder 203 is an electric circuit that decodes the attribute information. For example, the attribute information decoder 203 decodes the attribute information from the bitstream in accordance with a format defined for the attribute information.

[0098] The vertex information, connection information, and attribute information may be decoded using variable length decoding or fixed length decoding, which may correspond to Huffman coding, context-adaptive binary arithmetic coding (CABAC), or the like.

[0099] The vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be integrated together, or each of the vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be further subdivided into multiple components.

[0100] 8 is a block diagram showing another example of the configuration of the decoding device 200 according to this embodiment. For example, the decoding device 200 includes a pre-processor 204 and a post-processor 205 in addition to the configuration shown in FIG.

[0101] The preprocessor 204 is an electrical circuit that performs processing before decoding the vertex information, connection information, and attribute information. For example, the preprocessor 204 may perform conversion processing, separation processing, multiplexing processing, or the like on the bitstream before decoding the vertex information, connection information, and attribute information.

[0102] More specifically, for example, the preprocessor 204 may separate a sub-bitstream corresponding to vertex information, a sub-bitstream corresponding to connectivity information, and a sub-bitstream corresponding to attribute information from the bitstream. Also, for example, the preprocessor 204 may perform variable-length decoding on the bitstream in advance before decoding the vertex information, connectivity information, and attribute information.

[0103] The post-processor 205 is an electrical circuit that performs processing after the vertex information, connection information, and attribute information are decoded. For example, the post-processor 205 may perform conversion processing, separation processing, multiplexing processing, or the like on the decoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 205 may multiplex the decoded vertex information, connection information, and attribute information onto a three-dimensional mesh.

[0104] <Bitstream> Vertex information, connection information, and attribute information are coded and stored in a bitstream. The relationship between this information and the bitstream is shown below.

[0105] 9 is a conceptual diagram showing an example of the configuration of a bitstream according to this embodiment. In this example, connection information, vertex information, and attribute information are integrated in the bitstream. For example, the connection information, vertex information, and attribute information may be included in a single file.

[0106] Furthermore, multiple portions of this information may be stored sequentially, such as a first portion of connection information, a first portion of vertex information, a first portion of attribute information, a second portion of connection information, a second portion of vertex information, a second portion of attribute information, etc. These multiple portions may correspond to multiple portions that are different in time, multiple portions that are different in space, or multiple different faces.

[0107] Furthermore, the storage order of the connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0108] 10 is a conceptual diagram showing another example of the configuration of a bitstream according to this embodiment. In this example, a plurality of files are included in the bitstream, and connection information, vertex information, and attribute information are stored in different files. Here, a file containing connection information, a file containing vertex information, and a file containing attribute information are shown, but the storage format is not limited to this example. For example, two types of information among the connection information, vertex information, and attribute information may be included in one file, and the remaining type of information may be included in another file.

[0109] Alternatively, the information may be split and stored in more files. For example, multiple pieces of connectivity information may be stored in multiple files, multiple pieces of vertex information may be stored in multiple files, or multiple pieces of attribute information may be stored in multiple files. These multiple pieces may correspond to multiple temporally different pieces, multiple spatially different pieces, or multiple different faces.

[0110] Furthermore, the storage order of the connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0111] 11 is a conceptual diagram showing another example of the configuration of a bitstream according to this embodiment. In this example, the bitstream is composed of multiple separable sub-bitstreams, and connection information, vertex information, and attribute information are stored in different sub-bitstreams.

[0112] Here, a sub-bitstream containing connection information, a sub-bitstream containing vertex information, and a sub-bitstream containing attribute information are shown, but the storage format is not limited to this example.

[0113] For example, two types of information among the connection information, vertex information, and attribute information may be included in one sub-bitstream, and the remaining type of information may be included in another sub-bitstream. Specifically, attribute information of a two-dimensional image or the like may be stored in a sub-bitstream that complies with an image coding method, separate from the sub-bitstreams of the connection information and vertex information.

[0114] Also, each sub-bitstream may include multiple files, and multiple pieces of connectivity information may be stored in multiple files, multiple pieces of vertex information may be stored in multiple files, or multiple pieces of attribute information may be stored in multiple files.

[0115] 9, 10, and 11, and a storage order different from the above examples may be used. For example, the vertex information, connection information, and attribute information may be stored in the bitstream in this order. Alternatively, the connection information, connection information, and attribute information may be stored in the bitstream in any of the following orders: connection information, attribute information, and vertex information; vertex information, attribute information, and connection information; attribute information, connection information, and vertex information; or attribute information, vertex information, and connection information.

[0116] Furthermore, each of the connection information, vertex information, and attribute information may be divided into a plurality of data, and the plurality of data may be stored in a cyclical or random order within the bitstream.

[0117] 12 is a block diagram showing a specific example of an encoding / decoding system according to this embodiment. In FIG. 12, the encoding / decoding system includes a three-dimensional data encoding system 110, a three-dimensional data decoding system 210, and an external connector 310.

[0118] The three-dimensional data encoding system 110 includes a controller 111, an input / output processor 112, a three-dimensional data encoder 113, a three-dimensional data generator 115, and a system multiplexer 114. The three-dimensional data decoding system 210 includes a controller 211, an input / output processor 212, a three-dimensional data decoder 213, a system demultiplexer 214, a presenter 215, and a user interface 216.

[0119] In the three-dimensional data encoding system 110, sensor data is input from a sensor terminal to a three-dimensional data generator 115. The three-dimensional data generator 115 generates three-dimensional data, such as point cloud data or mesh data, from the sensor data and inputs it to a three-dimensional data encoder 113.

[0120] For example, the three-dimensional data generator 115 generates vertex information, and generates connection information and attribute information corresponding to the vertex information. The three-dimensional data generator 115 may process the vertex information when generating the connection information and attribute information. For example, the three-dimensional data generator 115 may reduce the amount of data by deleting duplicate vertices, or may transform the vertex information (such as by shifting its position, rotating it, or normalizing it). The three-dimensional data generator 115 may also render the attribute information.

[0121] Furthermore, although the three-dimensional data generator 115 is a component of the three-dimensional data encoding system 110 in FIG. 12, it may be arranged externally and independently of the three-dimensional data encoding system 110.

[0122] The sensor terminal that provides the sensor data for generating the three-dimensional data may be, for example, a moving body such as an automobile, a flying object such as an airplane, a mobile terminal, a camera, etc. Furthermore, a distance sensor such as a LIDAR, a millimeter wave radar, an infrared sensor, or a range finder, a stereo camera, or a combination of multiple monocular cameras may also be used as the sensor terminal.

[0123] The sensor data may be the distance (position) of the object, monocular camera images, stereo camera images, color, reflectance, sensor attitude, orientation, gyro, sensing position (GPS information or altitude), speed, acceleration, sensing time, temperature, air pressure, humidity, or magnetism.

[0124] The three-dimensional data encoder 113 corresponds to the encoding device 100 shown in FIG. 5 and other figures. For example, the three-dimensional data encoder 113 encodes three-dimensional data to generate encoded data. The three-dimensional data encoder 113 also generates control information when encoding the three-dimensional data. The three-dimensional data encoder 113 then inputs the encoded data together with the control information to the system multiplexer 114.

[0125] The encoding method for the three-dimensional data may be an encoding method using geometry or an encoding method using a video codec. Here, the encoding method using geometry may also be referred to as a geometry-based encoding method. The encoding method using a video codec may also be referred to as a video-based encoding method.

[0126] The system multiplexer 114 multiplexes the encoded data and control information input from the 3D data encoder 113 to generate multiplexed data using a specified multiplexing method. The system multiplexer 114 may multiplex other media such as video, audio, subtitles, application data, or document files, or reference time information, along with the encoded data and control information of the 3D data. Furthermore, the system multiplexer 114 may multiplex attribute information related to the sensor data or the 3D data.

[0127] For example, the multiplexed data may have a file format for storage or a packet format for transmission. As these formats, ISOBMFF or a format based on ISOBMFF may be used. Also, MPEG-DASH, MMT, MPEG-2 TS Systems, RTP, or the like may be used.

[0128] The multiplexed data is then output as a transmission signal to the external connector 310 by the input / output processor 112. The multiplexed data may be transmitted as a transmission signal by wire or wirelessly. Alternatively, the multiplexed data is stored in an internal memory or a storage device. The multiplexed data may be transmitted to a cloud server via the Internet or may be stored in an external storage device.

[0129] For example, the transmission or storage of the multiplexed data is performed by a method according to the medium for transmission or storage, such as broadcasting or communication. The communication protocol may be http, ftp, TCP, UDP, IP, or a combination thereof. Furthermore, a pull-type communication method or a push-type communication method may be used.

[0130] For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), coaxial cable, etc. may be used. For wireless transmission, 3GPP (registered trademark), 3G / 4G / 5G defined by IEEE, wireless LAN, Wi-Fi, Bluetooth, or millimeter wave may be used. For broadcasting, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 may be used.

[0131] The sensor data may be input to the three-dimensional data generator 115 or the system multiplexer 114. The three-dimensional data or encoded data may be output as a transmission signal directly to the external connector 310 via the input / output processor 112. The transmission signal output from the three-dimensional data encoding system 110 is input to the three-dimensional data decoding system 210 via the external connector 310.

[0132] Furthermore, each operation of the three-dimensional data encoding system 110 may be controlled by a controller 111 that executes an application program.

[0133] In the three-dimensional data decoding system 210, a transmission signal is input to an input / output processor 212. The input / output processor 212 decodes multiplexed data having a file format or a packet format from the transmission signal and inputs the multiplexed data to a system demultiplexer 214. The system demultiplexer 214 obtains coded data and control information from the multiplexed data and inputs them to a three-dimensional data decoder 213. The system demultiplexer 214 may extract other media or reference time information from the multiplexed data.

[0134] The three-dimensional data decoder 213 corresponds to the decoding device 200 shown in Fig. 7 etc. For example, the three-dimensional data decoder 213 decodes three-dimensional data from the encoded data based on a predefined encoding method. The three-dimensional data is then presented to the user by the presenter 215.

[0135] Additionally, additional information such as sensor data may be input to the presenter 215. The presenter 215 may present three-dimensional data based on the additional information. Additionally, a user instruction may be input from a user terminal to the user interface 216. Then, the presenter 215 may present three-dimensional data based on the input instruction.

[0136] The input / output processor 212 may acquire the three-dimensional data and the encoded data from the external connector 310 .

[0137] Furthermore, each operation of the three-dimensional data decoding system 210 may be controlled by a controller 211 that executes an application program.

[0138] 13 is a conceptual diagram showing an example of the configuration of point cloud data according to this embodiment. The point cloud data is data of a group of points representing a three-dimensional object.

[0139] Specifically, a point cloud is made up of a plurality of points, and has position information indicating the three-dimensional coordinate position of each point and attribute information indicating the attribute of each point. The position information is also expressed as geometry.

[0140] The type of attribute information may be, for example, color, reflectance, etc. One point may be associated with attribute information of one type, one point may be associated with attribute information of multiple different types, or one point may be associated with attribute information having multiple values ​​for the same type.

[0141] 14 is a conceptual diagram showing an example of a data file of point cloud data according to this embodiment. This example shows a case where there is a one-to-one correspondence between position information items and attribute information items, and shows position information and attribute information for N points that make up the point cloud data. In this example, the position information is information indicating a three-dimensional coordinate position using three axes, x, y, and z, and the attribute information is information indicating a color using RGB. A PLY file or the like can be used as a representative data file for point cloud data.

[0142] 15 is a conceptual diagram showing an example of the configuration of mesh data according to this embodiment. Mesh data is data used in CG (Computer Graphics) and the like, and is three-dimensional mesh data that shows the three-dimensional shape of an object using multiple surfaces. Each surface is also expressed as a polygon, and has a polygonal shape such as a triangle or a rectangle.

[0143] Specifically, a 3D mesh is composed of a plurality of points constituting a point cloud, as well as a plurality of edges and a plurality of faces. Each point is also expressed as a vertex or a position. Each edge corresponds to a line segment connected by two vertices. Each face corresponds to an area surrounded by three or more edges.

[0144] Furthermore, a three-dimensional mesh has position information indicating the three-dimensional coordinate positions of vertices. The position information is also expressed as vertex information or geometry. A three-dimensional mesh also has connection information indicating the relationship between multiple vertices that make up an edge or a face. The connection information is also expressed as connectivity. A three-dimensional mesh also has attribute information indicating the attributes of the vertices, edges, or faces. The attribute information in a three-dimensional mesh is also expressed as texture.

[0145] For example, the attribute information may indicate the color, reflectance, or normal vector for a vertex, edge, or face. The direction of the normal vector may represent the front and back of the face.

[0146] The mesh data may be stored in a data file format such as an object file.

[0147] 16 is a conceptual diagram showing an example of a data file of mesh data according to this embodiment. In this example, the data file includes position information G(1) to G(N) of N vertices that make up the three-dimensional mesh, and attribute information A1(1) to A1(N) of the N vertices. Also, in this example, M pieces of attribute information A2(1) to A2(M) are included. The attribute information items do not need to correspond one-to-one to vertices or faces. Furthermore, attribute information need not exist.

[0148] The connection information is represented by a combination of vertex indices. n[1, 3, 4] indicates a triangular face formed by three vertices, n=1, n=3, and n=4. Also, m[2, 4, 6] indicates that the attribute information of m=2, m=4, and m=6 corresponds to the three vertices, respectively.

[0149] Furthermore, the actual contents of the attribute information may be written in a separate file. A pointer to that content may be associated with a vertex, a face, or the like. For example, attribute information indicating an image for a face may be stored in a two-dimensional attribute map file. The file name of the attribute map and two-dimensional coordinate values ​​in the attribute map may be written in attribute information A2(1) to A2(M). The method of specifying attribute information for a face is not limited to these methods, and any method may be used.

[0150] 17 is a conceptual diagram showing types of three-dimensional data according to this embodiment. Point cloud data and mesh data may represent static objects or dynamic objects. A static object is an object that does not change over time, and a dynamic object is an object that changes over time. A static object may correspond to three-dimensional data for any point in time.

[0151] For example, point cloud data for a given point in time may be referred to as a PCC frame, mesh data for a given point in time may be referred to as a mesh frame, and PCC frames and mesh frames may be simply referred to as frames.

[0152] The area of ​​the object may be limited to a certain range, as in normal video data, or may not be limited, as in map data. The density of points or surfaces may be determined in various ways. Sparse point cloud data or sparse mesh data may be used, or dense point cloud data or dense mesh data may be used.

[0153] Next, encoding and decoding of a point cloud or a three-dimensional mesh will be described. The device, process, or syntax for encoding and decoding vertex information of a three-dimensional mesh in the present disclosure may be applied to encoding and decoding of a point cloud. The device, process, or syntax for encoding and decoding of a point cloud in the present disclosure may be applied to encoding and decoding vertex information of a three-dimensional mesh.

[0154] Furthermore, a device, process, or syntax for encoding and decoding attribute information of a point cloud in the present disclosure may be applied to encoding and decoding connectivity information or attribute information of a three-dimensional mesh.Furthermore, a device, process, or syntax for encoding and decoding connectivity information or attribute information of a three-dimensional mesh in the present disclosure may be applied to encoding and decoding attribute information of a point cloud.

[0155] Furthermore, at least some of the processing may be shared between the encoding and decoding of point cloud data and the encoding and decoding of mesh data, thereby reducing the scale of the circuit and software program.

[0156] 18 is a block diagram showing an example configuration of a three-dimensional data encoder 113 according to this embodiment. In this example, the three-dimensional data encoder 113 includes a vertex information encoder 121, an attribute information encoder 122, a metadata encoder 123, and a multiplexer 124. The vertex information encoder 121, the attribute information encoder 122, and the multiplexer 124 may correspond to the vertex information encoder 101, the attribute information encoder 103, the post-processor 105, etc. in FIG.

[0157] In this example, the three-dimensional data encoder 113 encodes the three-dimensional data according to a geometry-based encoding method, which takes into account the three-dimensional structure. In addition, in the geometry-based encoding method, attribute information is encoded using configuration information obtained in encoding the vertex information.

[0158] Specifically, first, vertex information, attribute information, and metadata included in three-dimensional data generated from sensor data are input to a vertex information encoder 121, an attribute information encoder 122, and a metadata encoder 123, respectively. Here, connectivity information included in the three-dimensional data may be treated in the same way as attribute information. In addition, in the case of point cloud data, position information may be treated as vertex information.

[0159] The vertex information encoder 121 encodes the vertex information into compressed vertex information and outputs the compressed vertex information as encoded data to the multiplexer 124. The vertex information encoder 121 also generates metadata for the compressed vertex information and outputs it to the multiplexer 124. The vertex information encoder 121 also generates configuration information and outputs it to the attribute information encoder 122.

[0160] The attribute information encoder 122 uses the configuration information generated by the vertex information encoder 121 to encode the attribute information into compressed attribute information and outputs the compressed attribute information as encoded data to the multiplexer 124. The attribute information encoder 122 also generates metadata of the compressed attribute information and outputs it to the multiplexer 124.

[0161] The metadata encoder 123 encodes compressible metadata into compressed metadata and outputs the compressed metadata as encoded data to the multiplexer 124. The metadata encoded by the metadata encoder 123 may be used to encode vertex information and attribute information.

[0162] The multiplexer 124 multiplexes the compressed vertex information, the compressed vertex information metadata, the compressed attribute information, the compressed attribute information metadata, and the compressed metadata into a bitstream, and then inputs the bitstream to the system layer.

[0163] 19 is a block diagram showing an example configuration of a three-dimensional data decoder 213 according to this embodiment. In this example, the three-dimensional data decoder 213 includes a vertex information decoder 221, an attribute information decoder 222, a metadata decoder 223, and a demultiplexer 224. The vertex information decoder 221, the attribute information decoder 222, and the demultiplexer 224 may correspond to the vertex information decoder 201, the attribute information decoder 203, the preprocessor 204, and the like in FIG.

[0164] In this example, the three-dimensional data decoder 213 decodes three-dimensional data according to a geometry-based encoding method. The three-dimensional structure is taken into consideration in the decoding according to the geometry-based encoding method. Furthermore, in the decoding according to the geometry-based encoding method, attribute information is decoded using configuration information obtained in decoding vertex information.

[0165] Specifically, first, a bitstream is input from the system layer to a demultiplexer 224. The demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream. The compressed vertex information and compressed vertex information metadata are input to a vertex information decoder 221. The compressed attribute information and compressed attribute information metadata are input to an attribute information decoder 222. The metadata is input to a metadata decoder 223.

[0166] The vertex information decoder 221 decodes vertex information from the compressed vertex information using metadata of the compressed vertex information. The vertex information decoder 221 also generates configuration information and outputs it to the attribute information decoder 222. The attribute information decoder 222 decodes attribute information from the compressed attribute information using the configuration information generated by the vertex information decoder 221 and the metadata of the compressed attribute information. The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 may be used to decode the vertex information and the attribute information.

[0167] Thereafter, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. Note that, for example, this metadata is metadata of the vertex information and attribute information, and can be used in an application program.

[0168] 20 is a block diagram showing another example configuration of the three-dimensional data encoder 113 according to the present embodiment. In this example, the three-dimensional data encoder 113 includes a vertex image generator 131, an attribute image generator 132, a metadata generator 133, a video encoder 134, a metadata encoder 123, and a multiplexer 124. The vertex image generator 131, the attribute image generator 132, and the video encoder 134 may correspond to the vertex information encoder 101 and the attribute information encoder 103 in FIG. 6 , etc.

[0169] In this example, the 3D data encoder 113 encodes the 3D data according to a video-based encoding method. In encoding according to the video-based encoding method, multiple 2D images are generated from the 3D data, and the multiple 2D images are encoded according to a video encoding method. Here, the video encoding method may be High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), or the like.

[0170] Specifically, first, vertex information and attribute information included in three-dimensional data generated from sensor data are input to a metadata generator 133. The vertex information and attribute information are then input to a vertex image generator 131 and an attribute image generator 132, respectively. The metadata included in the three-dimensional data is then input to a metadata encoder 123. Here, connectivity information included in the three-dimensional data may be treated in the same way as attribute information. In the case of point cloud data, position information may be treated as vertex information.

[0171] The metadata generator 133 generates map information of a plurality of two-dimensional images from the vertex information and attribute information, and inputs the map information to the vertex image generator 131, the attribute image generator 132, and the metadata encoder 123.

[0172] The vertex image generator 131 generates a vertex image based on the vertex information and map information, and inputs the generated image to the video encoder 134. The attribute image generator 132 generates an attribute image based on the attribute information and map information, and inputs the generated image to the video encoder 134.

[0173] The video encoder 134 encodes the vertex images and attribute images into compressed vertex information and compressed attribute information, respectively, in accordance with a video encoding method, and outputs the compressed vertex information and compressed attribute information as encoded data to the multiplexer 124. The video encoder 134 also generates metadata for the compressed vertex information and metadata for the compressed attribute information, and outputs them to the multiplexer 124.

[0174] The metadata encoder 123 encodes the compressible metadata into compressed metadata and outputs the compressed metadata as encoded data to the multiplexer 124. The compressible metadata includes map information. The metadata encoded by the metadata encoder 123 may also be used to encode vertex information and attribute information.

[0175] The multiplexer 124 multiplexes the compressed vertex information, the compressed vertex information metadata, the compressed attribute information, the compressed attribute information metadata, and the compressed metadata into a bitstream, and then inputs the bitstream to the system layer.

[0176] 21 is a block diagram showing another example configuration of the 3D data decoder 213 according to this embodiment. In this example, the 3D data decoder 213 includes a vertex information generator 231, an attribute information generator 232, a video decoder 234, a metadata decoder 223, and a demultiplexer 224. The vertex information generator 231, the attribute information generator 232, and the video decoder 234 may correspond to the vertex information decoder 201 and the attribute information decoder 203 in FIG. 8, etc.

[0177] In this example, the 3D data decoder 213 decodes the 3D data according to a video-based coding method. In the decoding according to the video-based coding method, a plurality of 2D images are decoded according to a video coding method, and 3D data is generated from the plurality of 2D images. Here, the video coding method may be High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), or the like.

[0178] Specifically, first, a bitstream is input from the system layer to the demultiplexer 224. The demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream. The compressed vertex information, compressed vertex information metadata, compressed attribute information, and compressed attribute information metadata are input to the video decoder 234. The compressed metadata is input to the metadata decoder 223.

[0179] The video decoder 234 decodes the vertex images in accordance with the video encoding method. At this time, the video decoder 234 decodes the vertex images from the compressed vertex information using the metadata of the compressed vertex information. Then, the video decoder 234 inputs the vertex images to the vertex information generator 231. The video decoder 234 also decodes the attribute images in accordance with the video encoding method. At this time, the video decoder 234 decodes the attribute images from the compressed attribute information using the metadata of the compressed attribute information. Then, the video decoder 234 inputs the attribute images to the attribute information generator 232.

[0180] The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 includes map information used to generate vertex information and attribute information. The metadata decoded by the metadata decoder 223 may also be used to decode vertex images and attribute images.

[0181] The vertex information generator 231 reproduces vertex information from the vertex image in accordance with the map information included in the metadata decoded by the metadata decoder 223. The attribute information generator 232 reproduces attribute information from the attribute image in accordance with the map information included in the metadata decoded by the metadata decoder 223.

[0182] Thereafter, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. Note that, for example, this metadata is metadata of the vertex information and attribute information, and can be used in an application program.

[0183] Fig. 22 is a conceptual diagram showing a specific example of encoding processing according to this embodiment. Fig. 22 shows a three-dimensional data encoder 113 and a description encoder 148. In this example, the three-dimensional data encoder 113 includes a two-dimensional data encoder 141 and a mesh data encoder 142. The two-dimensional data encoder 141 includes a texture encoder 143. The mesh data encoder 142 includes a vertex information encoder 144 and a connection information encoder 145.

[0184] The vertex information encoder 144, the connection information encoder 145, and the texture encoder 143 may correspond to the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 in FIG.

[0185] For example, the two-dimensional data encoder 141 operates as a texture encoder 143 and generates a texture file by encoding the texture corresponding to the attribute information as two-dimensional data according to an image encoding method or a video encoding method.

[0186] The mesh data encoder 142 also operates as a vertex information encoder 144 and a connectivity information encoder 145, and generates a mesh file by encoding the vertex information and connectivity information. The mesh data encoder 142 may further encode mapping information for textures. The encoded mapping information may then be included in the mesh file.

[0187] The description encoder 148 also generates a description file by encoding a description corresponding to metadata such as text data. The description encoder 148 may encode the description at the system layer. For example, the description encoder 148 may be included in the system multiplexer 114 of FIG. 12 .

[0188] The above operations generate a bitstream containing texture files, mesh files, and description files, which may be multiplexed into the bitstream in file formats such as glTF (Graphics Language Transmission Format) or USD (Universal Scene Description).

[0189] The three-dimensional data encoder 113 may include two mesh data encoders as the mesh data encoder 142. For example, one mesh data encoder encodes vertex information and connectivity information of a static three-dimensional mesh, and the other mesh data encoder encodes vertex information and connectivity information of a dynamic three-dimensional mesh.

[0190] Correspondingly, two mesh files may then be included in the bitstream, for example one mesh file corresponding to a static 3D mesh and another mesh file corresponding to a dynamic 3D mesh.

[0191] Furthermore, the static three-dimensional mesh may be a three-dimensional mesh of an intraframe coded using intraprediction, and the dynamic three-dimensional mesh may be a three-dimensional mesh of an interframe coded using interprediction. Furthermore, information on the dynamic three-dimensional mesh may be differential information between vertex information or connectivity information of the three-dimensional mesh of an intraframe and vertex information or connectivity information of the three-dimensional mesh of an interframe.

[0192] Fig. 23 is a conceptual diagram showing a specific example of the decoding process according to this embodiment. Fig. 23 shows a three-dimensional data decoder 213, a description decoder 248, and a renderer 247. In this example, the three-dimensional data decoder 213 includes a two-dimensional data decoder 241, a mesh data decoder 242, and a mesh reconstructor 246. The two-dimensional data decoder 241 includes a texture decoder 243. The mesh data decoder 242 includes a vertex information decoder 244 and a connectivity information decoder 245.

[0193] The vertex information decoder 244, the connection information decoder 245, the texture decoder 243, and the mesh reconstructor 246 may correspond to the vertex information decoder 201, the connection information decoder 202, the attribute information decoder 203, and the post-processor 205 in Fig. 8. The presenter 247 may correspond to the presenter 215 in Fig. 12.

[0194] For example, the two-dimensional data decoder 241 operates as a texture decoder 243, and decodes the texture corresponding to the attribute information from the texture file as two-dimensional data in accordance with an image coding method or a video coding method.

[0195] The mesh data decoder 242 also operates as a vertex information decoder 244 and a connectivity information decoder 245 to decode vertex information and connectivity information from the mesh file. The mesh data decoder 242 may further decode mapping information for textures from the mesh file.

[0196] The description decoder 248 also decodes descriptions corresponding to metadata such as text data from the description file. The description decoder 248 may decode the descriptions at the system layer. For example, the description decoder 248 may be included in the system demultiplexer 214 of FIG. 12 .

[0197] The mesh reconstructor 246 reconstructs a 3D mesh from the vertex information, connectivity information, and textures according to the description. The renderer 247 renders and outputs the 3D mesh according to the description.

[0198] Through the above operations, a 3D mesh is reconstructed and output from a bitstream containing a texture file, a mesh file, and a description file.

[0199] The three-dimensional data decoder 213 may include two mesh data decoders as the mesh data decoder 242. For example, one mesh data decoder decodes vertex information and connectivity information of a static three-dimensional mesh, and the other mesh data decoder decodes vertex information and connectivity information of a dynamic three-dimensional mesh.

[0200] Correspondingly, two mesh files may then be included in the bitstream, for example one mesh file corresponding to a static 3D mesh and another mesh file corresponding to a dynamic 3D mesh.

[0201] Furthermore, the static three-dimensional mesh may be a three-dimensional mesh of an intraframe coded using intraprediction, and the dynamic three-dimensional mesh may be a three-dimensional mesh of an interframe coded using interprediction. Furthermore, information on the dynamic three-dimensional mesh may be differential information between vertex information or connectivity information of the three-dimensional mesh of an intraframe and vertex information or connectivity information of the three-dimensional mesh of an interframe.

[0202] A dynamic 3D mesh coding method is sometimes called DMC (Dynamic Mesh Coding), and a video-based dynamic 3D mesh coding method is sometimes called V-DMC (Video-based Dynamic Mesh Coding).

[0203] The point cloud encoding method is sometimes called PCC (Point Cloud Compression). The point cloud video-based encoding method is sometimes called V-PCC (Video-based Point Cloud Compression). The point cloud geometry-based encoding method is sometimes called G-PCC (Geometry-based Point Cloud Compression).

[0204] <Implementation Example> Fig. 24 is a block diagram showing an implementation example of the encoding device 100 according to this embodiment. The encoding device 100 includes a circuit 151 and a memory 152. For example, multiple components of the encoding device 100 shown in Fig. 5 etc. are implemented by the circuit 151 and memory 152 shown in Fig. 24.

[0205] The circuit 151 is a circuit that performs information processing and is a circuit that can access the memory 152. For example, the circuit 151 is a dedicated or general-purpose electric circuit that encodes a three-dimensional mesh. The circuit 151 may be a processor such as a CPU. Alternatively, the circuit 151 may be a collection of multiple electric circuits.

[0206] The memory 152 is a dedicated or general-purpose memory that stores information used by the circuit 151 to encode the three-dimensional mesh. The memory 152 may be an electric circuit and may be connected to the circuit 151. The memory 152 may also be included in the circuit 151. The memory 152 may also be a collection of multiple electric circuits. The memory 152 may also be a magnetic disk, an optical disk, or the like, and may also be expressed as a storage, a recording medium, or the like. The memory 152 may also be a non-volatile memory or a volatile memory.

[0207] For example, the memory 152 may store a three-dimensional mesh or a bitstream, or may store a program for the circuit 151 to encode the three-dimensional mesh.

[0208] Note that the encoding device 100 does not necessarily have to implement all of the components shown in Figure 5 and the like, and does not necessarily have to perform all of the processes shown here. Some of the components shown in Figure 5 and the like may be included in another device, and some of the processes shown here may be executed by another device. Furthermore, the encoding device 100 may implement any combination of the components of the present disclosure, and may perform any combination of the processes of the present disclosure.

[0209] Fig. 25 is a block diagram showing an example implementation of a decoding device 200 according to this embodiment. The decoding device 200 includes a circuit 251 and a memory 252. For example, multiple components of the decoding device 200 shown in Fig. 7 and other figures are implemented by the circuit 251 and memory 252 shown in Fig. 25.

[0210] The circuit 251 is a circuit that performs information processing and is a circuit that can access the memory 252. For example, the circuit 251 is a dedicated or general-purpose electric circuit that decodes a three-dimensional mesh. The circuit 251 may be a processor such as a CPU. Alternatively, the circuit 251 may be a collection of multiple electric circuits.

[0211] The memory 252 is a dedicated or general-purpose memory that stores information for the circuit 251 to decode the 3D mesh. The memory 252 may be an electric circuit and may be connected to the circuit 251. The memory 252 may also be included in the circuit 251. The memory 252 may also be a collection of multiple electric circuits. The memory 252 may also be a magnetic disk, an optical disk, or the like, and may also be expressed as a storage, a recording medium, or the like. The memory 252 may also be a non-volatile memory or a volatile memory.

[0212] For example, the memory 252 may store a three-dimensional mesh or a bitstream, or may store a program for the circuit 251 to decode the three-dimensional mesh.

[0213] Note that the decoding device 200 does not necessarily have to implement all of the components shown in Figure 7 and the like, and does not necessarily have to perform all of the processes shown here. Some of the components shown in Figure 7 and the like may be included in another device, and some of the processes shown here may be executed by another device. Furthermore, the decoding device 200 may implement any combination of the components of the present disclosure, and may perform any combination of the processes of the present disclosure.

[0214] The encoding method and the decoding method including the steps performed by each component of the encoding device 100 and the decoding device 200 of the present disclosure may be executed by any device or system. For example, part or all of the encoding method and the decoding method may be executed by a computer including a processor, a memory, an input / output circuit, etc. In this case, the encoding method and the decoding method may be executed by the computer executing a program for causing the computer to execute the encoding method and the decoding method.

[0215] Alternatively, the program or the bitstream may be recorded on a non-transitory computer-readable recording medium such as a CD-ROM.

[0216] An example of a program may be a bitstream. For example, a bitstream including an encoded three-dimensional mesh includes syntax elements for causing the decoding device 200 to decode the three-dimensional mesh. The bitstream then causes the decoding device 200 to decode the three-dimensional mesh according to the syntax elements included in the bitstream. Thus, the bitstream may play a role similar to that of a program.

[0217] The bitstream may be an encoded bitstream containing the encoded 3D mesh, or may be a multiplexed bitstream containing the encoded 3D mesh and other information.

[0218] Furthermore, each component of the encoding device 100 and the decoding device 200 may be configured with dedicated hardware, general-purpose hardware that executes the above-mentioned programs, or a combination of these. The general-purpose hardware may be configured with a memory in which the programs are recorded and a general-purpose processor that reads and executes the programs from the memory. Here, the memory may be a semiconductor memory or a hard disk, and the general-purpose processor may be a CPU.

[0219] Furthermore, the dedicated hardware may be configured with a memory, a dedicated processor, etc. For example, the dedicated processor may execute the encoding method and the decoding method by referring to a memory for recording data.

[0220] Furthermore, as described above, each component of the encoding device 100 and the decoding device 200 may be an electric circuit. These electric circuits may form a single electric circuit as a whole, or each may be a separate electric circuit. Furthermore, these electric circuits may correspond to dedicated hardware, or may correspond to general-purpose hardware that executes the above-mentioned programs, etc. Furthermore, the encoding device 100 and the decoding device 200 may be implemented as an integrated circuit.

[0221] Furthermore, the encoding device 100 may be a transmitting device that transmits the three-dimensional mesh, and the decoding device 200 may be a receiving device that receives the three-dimensional mesh.

[0222] Displacement Encoding and Decoding The following terminology is used here by way of example:

[0223] (1) Image An image is a data unit made up of a set of pixels, and includes a picture or a block smaller than a picture. Images include both moving images and still images.

[0224] (2) Picture A picture is a unit of image processing that is made up of a set of pixels, and is also called a frame or field.

[0225] (3) Block A block is a processing unit consisting of a specific number of pixels. The term shown in the following example is also used for a block. The shape of a block is not particularly limited. A block may be, for example, a rectangular shape of M×N pixels or a square shape of M×M pixels. A block may also be a triangular shape, a circular shape, or another shape. Examples of blocks are as follows:

[0226] Slice, tile, or brick CTU, superblock, or basic division unit VPDU, processing division unit for hardware CU, processing block unit, prediction block unit (PU), or orthogonal transform block unit (TU) Sub-block

[0227] (4) Pixel or Sample A pixel or sample is the smallest point of an image, in other words, the smallest unit. Pixels or samples include not only pixels at integer positions, but also pixels at sub-pixel positions generated based on pixels at integer positions.

[0228] (5) Pixel Value or Sample Value: A pixel value or sample value is a unique value of a pixel. The pixel value or sample value may include a luma value, a chroma value, or an RGB gradation level, and may also include a depth value or a binary value of 0 or 1.

[0229] (6) Flags A flag indicates one or more bits. A flag is, for example, a parameter or index represented by two or more bits. A flag may indicate not only a value represented by a binary number, but also a value represented by a number other than a binary number.

[0230] (7) Signal: A signal is something that is symbolized or coded to transmit information. A signal includes a discrete digital signal or a continuous analog signal.

[0231] (8) Stream or Bit Stream A stream or bit stream is a digital data sequence that indicates the flow of digital data. A stream or bit stream may be a single stream, or may be configured to include multiple streams with multiple layers. A stream or bit stream may be transmitted by serial communication using a single transmission path, or may be transmitted by packet communication using multiple transmission paths.

[0232] (9) Difference: For scalar quantities, difference can include simple difference (x - y) and difference calculations, such as absolute difference (|x - y|), squared difference (x^2 - y^2), square root difference (√(x - y)), weighted difference (ax - b, where a and b are constants), or offset difference (x - y + a, where a is an offset).

[0233] (10) Sum. For scalar quantities, sums can include simple sum (x + y) and addition operations. Sum can also include absolute sum (|x + y|), sum of squares (x^2 + y^2), square root of sum (√(x + y)), weighted sum (ax + by, where a and b are constants), or offset sum (x + y + a, where a is an offset).

[0234] (11) "Based on" The expression "based on something" means that something other than that "something" may be taken into consideration. Also, "based on" can be used both when a direct result is obtained and when a result is obtained through an intermediate result.

[0235] (12) "Used" or "Using" The phrases "something was used" or "used something" mean that something other than the "something" may be taken into consideration. The phrases "used" or "used" may be used both in cases where a direct result is obtained and in cases where a result is obtained via an intermediate result.

[0236] (13) Prohibition "Prohibit" can be rephrased as "not permitted." Also, "not prohibited / prohibited" or "permitted / permitted" does not necessarily mean "obligation."

[0237] (14) "Restriction" or "Limitation" "Restriction" or "Limitation" can be rephrased as "not permitted / not allowed" or "not permitted / permitted." Furthermore, "prohibited / not prohibited" or "not permitted / permitted" does not necessarily mean "obligation." Furthermore, what is prohibited quantitatively or qualitatively may be either partial or total.

[0238] (15) Chroma The term chroma is an adjective, represented by the symbols Cb or Cr, that indicates that a sample array or a single sample represents one of the two color-difference signals associated with a primary color. The term chroma is sometimes used instead of the term chrominance.

[0239] (16) Luma The term luma is an adjective, represented by the symbols or subscripts Y or L, that indicates that a sample array or a single sample represents a monochrome signal for a primary color. The term luma is sometimes used instead of the term luminance.

[0240] The encoding / decoding system of this embodiment will be described below.

[0241] A typical three-dimensional model (also called a 3D model) digitally represents an object so that a user can explore the model using zoom, pan, and rotation in all three dimensions while it is rendered over time. One way to construct such a representation is to build a 3D mesh using triangles. The model stores the positions of the triangle vertices, their connectivity to each other, and their associated attributes (such as normals or UV patches).

[0242] Storing all this information in uncompressed form requires a very large storage space and therefore a very large bandwidth for transmission. The triangles that form the mesh often have repeating patterns and similar properties, especially in temporal and spatial neighborhoods. These repetitions can be exploited to develop efficient encoding and decoding methods for storage and transmission. One such encoding and decoding method is Video-based Dynamic Mesh Coding (V-DMC).

[0243] 26 is a block diagram showing another example of the configuration of the encoding / decoding system according to this embodiment. As shown in FIG. 26, the encoding / decoding system includes an encoding device 100 and a decoding device 200.

[0244] The encoding / decoding system accepts input three-dimensional meshes (also called 3D meshes) in the form of three-dimensional coordinates of vertices (vertex information), connectivity (connection information) and associated attributes (attribute information), which may include texture maps as well as geometry.

[0245] The encoding device 100 takes an input 3D mesh (also referred to as an input 3D mesh or input mesh) in the form of three-dimensional coordinates of vertices, connectivity, and associated attributes. The encoding device 100 encodes all associated information into a stream. The stream may consist of a single bitstream or multiple bitstreams.

[0246] The network 300 transmits the stream generated by the encoding device to the decoding device 200. The network 300 may be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof. Furthermore, the network 300 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting or satellite broadcasting. Instead of the network 300, a recording medium such as a digital versatile disc (DVD) or a blue-ray disc (BD) on which a stream is recorded may be used.

[0247] The stream is transmitted to a decoding device 200 via a network 300. The decoding device 200 decodes the bitstream and generates a 3D mesh using the 3D coordinates, connectivity, and associated attributes of the decoded vertices. The decoding device 200 outputs the generated 3D mesh (also referred to as an output 3D mesh or output mesh).

[0248] FIG. 27 is a diagram showing another example of the configuration of the encoding device 100.

[0249] As shown in FIG. 27, the encoding device 100 includes a preprocessor 1102 and a compressor 1107 .

[0250] The encoding device 100 reads an input mesh 1101 and passes it to a preprocessor 1102. The preprocessor 1102 processes the input mesh 1101 to extract a base mesh 1103, displacement data 1104, texture data 1105, and an attribute map 1106. The encoding device 100 passes the extracted base mesh 1103, displacement data 1104, texture data 1105, and attribute map 1106 to a compressor 1107.

[0251] The compressor 1107 can compress the base mesh 1103 , the displacement data 1104 , the texture data 1105 and the attribute map 1106 to generate a bitstream 1108 that can be sent to the decoding device 200 .

[0252] FIG. 28 is a diagram showing another example of the configuration of the decoding device 200.

[0253] As shown in FIG. 28, the decoding device 200 includes a decompressor 2102 and a post-processor 2107 .

[0254] The decoding device 200 reads a bitstream 2101 and passes it to a decompressor 2102. The decompressor 2102 decompresses a base mesh 2103, displacement data 2104, texture data 2105, and an attribute map 2106 from the bitstream 2101, and passes them to a post-processor 2107. An example of the displacement data 2104 is a displacement vector.

[0255] The post-processor 2107 processes the base mesh 2103 according to the displacement data 2104 and the attribute map 2106 to generate an output mesh 2108. The post-processor 2107 may further use information from the texture data 2105 to generate the output mesh 2108.

[0256] The encoding process performed by the encoding device 100 will be described in detail below.

[0257] Fig. 29 is a flow diagram showing the processing of the encoding device 100. Fig. 30 is an explanatory diagram conceptually showing the encoding of mesh frames. The processing of the encoding device 100 will be described with reference to Figs. 29 and 30.

[0258] In step S101, the encoding device 100 reads a three-dimensional mesh frame that is an input mesh frame. The input mesh frame is a mesh frame that is input to the encoding device 100. An example of a three-dimensional mesh frame that is an input mesh frame is shown as mesh frame 1301 (see FIG. 30 ).

[0259] In step S102, the encoding device 100 performs a decimation process on the input mesh frame read in step S101 to generate a base mesh frame having fewer vertices than the input mesh frame. The base mesh frame generated by decimating the mesh frame 1301 is shown as a base mesh frame 1302 (see FIG. 30 ). Note that the vertices of the base mesh frame 1302 may be located at positions different from those of the mesh frame 1301. Note that the base mesh frame 1302 may be subdivided by adding new vertices between existing vertices.

[0260] In step S103, the encoding device 100 calculates displacement information that the decoding device 200 uses to reconstruct a mesh frame. The displacement information corresponds to a displacement vector directed from a vertex of the base mesh frame generated in step S102 to a vertex of the input mesh frame. One method for calculating the displacement information is to subtract the coordinates of the vertex of the base mesh frame from the coordinates of the vertex of the input mesh frame. The displacement information calculated from the mesh frame 1301 and the base mesh frame 1302 is shown as displacement information 1303 (see FIG. 30 ). The displacement information 1303 is in vector format, in other words, expressed as a displacement vector. Note that the displacement information is converted into wavelet coefficients by wavelet transform, and then the wavelet coefficients can be assigned to planes of a video frame and encoded using a video codec.

[0261] In step S104, the encoding device 100 encodes the base mesh frame generated in step S102, the displacement information generated in step S103, the attribute map, and the texture data into a bitstream (corresponding to a compressed bitstream). An example of the bitstream is shown as bitstream 1304 (see FIG. 30).

[0262] Specifically, the bitstream 1304 includes vertex coordinates and connectivity information for vertices A, C, E, and F, displacement information, a video bitstream including texture data, and a compressed attribute map (see FIG. 30). The displacement information includes displacement information for displacing vertices based on vertex coordinates obtained from the subdivided base mesh frame. The compressed attribute map is texture coordinates for applying texture data to a mesh frame reconstructed using the base mesh frame and the displacement information.

[0263] The decoding process performed by the decoding device 200 will be described in detail below.

[0264] Fig. 31 is a flow diagram showing the processing of the decoding device 200. Fig. 32 is an explanatory diagram conceptually showing the decoding of a three-dimensional mesh. The processing of the decoding device 200 will be described with reference to Figs. 31 and 32.

[0265] In step S201, the decoding device 200 decodes a base mesh frame from a bitstream (corresponding to a compressed bitstream). An example of the decoded base mesh frame (corresponding to a decoded base mesh frame) is shown as a decoded base mesh frame 2301 (see FIG. 32).

[0266] In step S202, the decoding device 200 generates new vertices between existing vertices by subdivision of the base mesh frame decoded in step S201. An example of a mesh frame including the subdivided vertices is shown as mesh frame 2302 (see FIG. 32). At this point, the mesh frame 2302 includes all the vertices and connectivity relationships of the original mesh frame. Note that the positions of the vertices included in the mesh frame 2302 may differ from the corresponding vertices included in the input mesh frame.

[0267] In step S203, the decoding device 200 decodes the disparity information from the bitstream (corresponding to the compressed bitstream). An example of the decoded disparity information is shown as disparity information 2303 (see FIG. 32). The disparity information 2303 is in vector format, in other words, expressed as a disparity vector. At this time, the decoding device 200 decodes the wavelet coefficients using a video decoder and performs an inverse wavelet transform to restore the disparity vector.

[0268] In step S204, the decoding device 200 reconstructs the shape of the mesh frame by moving the vertices of the base mesh frame, including the subdivided vertices, to new positions using the displacement information. The decoding device 200 further reconstructs the decoded 3D mesh by applying texture data to the faces generated by the vertices and connectivity information of the mesh frame. An example of a reconstructed mesh frame is shown as mesh frame 2304 (see FIG. 32).

[0269] A method for predicting the coordinates of vertices according to this embodiment will be described below.

[0270] 33, 34, and 35 are explanatory diagrams showing examples of methods for predicting vertex coordinates according to this embodiment.

[0271] Specifically, Figures 33, 34, and 35 are explanatory diagrams of a method for predicting the coordinates of the vertices of a triangle in the encoding or decoding process of a 3D mesh composed of multiple triangles. In this example, the encoding device or decoding device predicts the coordinates of the vertices to be encoded or decoded using triangles and connection information that indicates how the vertices of the triangles are connected.

[0272] In FIG. 33 , vertices A1, B1, and C1 are vertices whose coordinates have already been coded or decoded. Vertex D is the vertex whose coordinates will be coded or decoded next. The coordinates of vertex E1 are predicted values ​​of the coordinates of vertex D, and are set to coordinates such that the rectangle A1B1E1C1 becomes a parallelogram. For example, the coding device or decoding device may derive vector A1E1 as the sum of vector A1C1 and vector A1B1, and code or decode vector E1D as a prediction residual. Note that the coordinates of vertex E1 may also be referred to as predicted value E1. The same applies to vertices E, E2, E3, etc., described below.

[0273] FIG. 34 shows an example in which a plurality of predicted values ​​E1, E2, and E3 are determined using the prediction method shown in FIG. 33, and the average value of the plurality of predicted values ​​E1, E2, and E3 is used as the predicted value of the coordinates of vertex D.

[0274] The encoding device or decoding device may derive vector AiEi (where i = 1, 2, 3) as the sum of vector AiCi and vector AiBi, determine the coordinates of vertex E using the average vector A1E of vectors A1E1, A1E2, and A1E3, and encode or decode vector ED as a prediction residual.

[0275] In addition, if all vertices of a triangle AiBiCi having an opposite side BiCi of a vertex D have been coded or decoded, the prediction method shown in FIG. 33 can be applied, and the predicted value Ei can be determined by this prediction method.

[0276] The average value of the multiple predicted values ​​E1, E2, and E3 may be an average value calculated by weighting the multiple predicted values ​​E1, E2, and E3. For example, the weights of the multiple predicted values ​​E1, E2, and E3 may be determined according to the magnitudes of vectors A1E1, A2E2, and A3E3, respectively. More generally, the weight of the predicted value Ei may be determined according to the magnitude of vector AiEi.

[0277] For example, since it is assumed that the smaller the magnitude of vector AiEi (i.e., the closer vertices Ai and Ei are, in other words, the smaller the distance between vertices Ai and Ei), the higher the accuracy of the prediction, the inverse of the magnitude of vector AiEi can be used as the weight of the predicted value Ei.

[0278] Furthermore, since it is assumed that the larger the magnitude of the vector AiEi, the less irregularities there are on the mesh surface and the higher the prediction accuracy, the magnitude of the vector AiEi can be used as the weight of the predicted value Ei.

[0279] It is assumed that the characteristics on which the weight of the predicted value Ei is determined will differ depending on the object represented by the mesh. Therefore, the method of determining the weight of the predicted value Ei may be switched for each sequence, frame, sub-mesh, or vertex.

[0280] Although this example shows an example in which three predicted values ​​E1, E2, and E3 are used, the number of predicted values ​​does not necessarily have to be three, and the number of predicted values ​​may be set according to the number of triangles AiBiCi to which the prediction method shown in Fig. 33 can be applied. The number of predicted values ​​may be one.

[0281] Furthermore, it is not necessary for all triangles AiBiCi to which the prediction method shown in Figure 33 can be applied to be used in determining the predicted value, and a predetermined number of triangles AiBiCi or less may be used in determining the predicted value. Note that, when the number of triangles AiBiCi to which the prediction method shown in Figure 33 can be applied is 0, the coordinates of a vertex located near vertex Bi or Ci, etc., that has been coded or decoded may be used as the predicted value. Furthermore, in the above case, when there is no coded or decoded vertex near vertex Bi or Ci, etc., predetermined coordinates such as the origin (0,0,0) may be used as the predicted value.

[0282] In FIG. 35 , vertices A1, B1, C1, F1, and G1 are vertices whose coordinates have already been coded or decoded. Here, vertices F1 and G1 are often points that are not on the same plane as triangle A1B1C1. Vertex D is the vertex whose coordinates are to be coded or decoded next. The coordinates of vertex E1 are predicted values ​​of the coordinates of vertex D, and are set to coordinates where quadrangle A1B1′E1C1′ forms a parallelogram. For example, the encoding or decoding device may derive vector A1E1 as the sum of vector A1C1′ and vector A1B1′, and encode or decode vector E1D as a prediction residual. The coordinates of vertex B1′ are set to coordinates where quadrangle A1B1′C1G1 forms a parallelogram. For example, vector A1B1′ is derived as the difference between vector A1C1 and vector A1G1. The coordinates of vertex C1' are set to coordinates that make quadrangle A1F1B1C1' a parallelogram. For example, vector A1C1' is derived as the difference between vector A1B1 and vector A1F1. Compared to the prediction method shown in FIG. 33 , by performing prediction using vertices F1 and G1, which are often not on the same plane as triangle A1B1C1, points that are not on plane A1B1C1 are set as predicted values, which may improve the prediction accuracy of the coordinates of vertices of a three-dimensional mesh with an uneven surface.

[0283] 34, similarly to the case where a plurality of predicted values ​​are determined using the prediction method shown in Fig. 33, a plurality of predicted values ​​Ei may be determined using the prediction method shown in Fig. 35, and the average value of the plurality of predicted values ​​Ei may be used as the predicted value of the coordinates of vertex D. When a triangle having a side BiCi opposite to vertex D is defined as triangle AiBiCi, a triangle sharing side AiBi with triangle AiBiCi is defined as triangle AiBiFi, and a triangle sharing side AiCi with triangle AiBiCi is defined as triangle AiCiGi, if vertices Ai, Bi, Ci, Fi, and Gi have all been encoded or decoded, the prediction method shown in Fig. 35 can be applied, and a predicted value Ei can be determined.

[0284] Note that, when vertices Ai, Bi, Ci, and Fi have all been coded or decoded, and vertex Gi has not been coded or decoded, vertex Bi may be used instead of vertex Bi', making the prediction method shown in FIG. 35 applicable. Also, when vertices Ai, Bi, Ci, and Gi have all been coded or decoded, and vertex Fi has not been coded or decoded, vertex Ci may be used instead of vertex Ci', making the prediction method shown in FIG. 35 applicable. The number of predicted values ​​may be set according to the number of triangles AiBiCi to which the prediction method shown in FIG. 35 can be applied. The number of predicted values ​​may be 1. Furthermore, it is not necessary for all triangles AiBiCi to which the prediction method shown in FIG. 35 can be applied to be used in determining predicted values; a predetermined number or less of triangles AiBiCi may be used in determining predicted values.

[0285] The encoding device or decoding device may predict the coordinates of the vertices of a triangle by fixedly applying one of the prediction methods described above, or may assign a unique index value to each method and notify the decoding device of the index value of the prediction method used for encoding for each vertex or for each group of vertices. In this case, the decoding device may switch prediction methods depending on the index value during decoding to determine the predicted values ​​of the coordinates of the vertices of the triangle. By notifying the decoding device of the index value of the prediction method used for encoding, the 3D mesh can be encoded while selecting a more effective prediction method, potentially reducing the amount of code. Note that the information notified from the encoding device to the decoding device may be any information that can identify the prediction method used for encoding, and does not necessarily have to be an index value. Furthermore, when determining whether vertices Ai, Bi, Ci, Fi, and Gi have been encoded or decoded, the encoding device or decoding device may determine that the vertices have not been encoded or decoded if there is no vertex corresponding to that position, such as if there is a hole in the 3D mesh.

[0286] FIG. 36 is an explanatory diagram showing an example of a method for predicting the coordinates of a vertex according to this embodiment.

[0287] In addition to the vertices Pa and Pb, the encoding device or decoding device may also use the black circles shown in Figure 36, i.e., the coordinates of the points that internally or externally divide the line segment PaPb, as candidates for predicted values. Here, the vertex Pa is the average value of at least one or more predicted values ​​Ei determined using the prediction method shown in Figure 34. The vertex Pb in Figure 34 is the average value of at least one or more predicted values ​​Ei determined using the prediction method shown in Figure 35, just as at least one or more predicted values ​​are determined using the prediction method shown in Figure 33.

[0288] For example, as illustrated in FIG. 36 , the encoding device or decoding device may set predicted value candidates at coordinates based on ¼ of the line segment PaPb. In other words, the encoding device or decoding device may set predicted value candidates at the coordinates of points that divide the line PaPb into segments each having a length equivalent to ¼ of the line segment PaPb. Note that the above reference is not limited to ¼ of the line segment PaPb, but may also be ½, etc. In other words, the encoding device or decoding device may set only coordinates based on ½ of the line segment PaPb as predicted value candidates. For example, the encoding device or decoding device may set five points, Pb, (Pa + Pb) / 2, Pa, (3×Pa − Pb) / 2, and 2×Pa − Pb, as predicted value candidates. Alternatively, the encoding device or decoding device may set four points, Pb, (Pa + Pb) / 2, Pa, and (3×Pa − Pb) / 2, as predicted value candidates. In addition, the determination of candidate predicted values ​​does not necessarily have to be based on the coordinates of 1 / 4 or 1 / 2 of the line segment PaPb, and candidate predicted values ​​may be set on the line PaPb using a common method predetermined by the encoding device and the decoding device.

[0289] FIG. 37 is an explanatory diagram showing an example of a method for predicting the coordinates of a vertex according to this embodiment.

[0290] 37, the encoding device or decoding device may set, in addition to vertex E1, multiple candidates for predicted values ​​such as vertices E11, E12, E13, and E14 on perpendicular lines to the parallelogram A1B1E1C1 that passes through vertex E1 in the prediction method shown in Fig. 33. Note that the length of line segment E1E1j (where j = 1, 2, 3, 4) may be determined based on the length of line segment A1E1 or line segment B1C1, etc.

[0291] For example, the encoding device or decoding device may set the length of line segment E1E1j to be k times the length of line segment A1E1 (e.g., k = 1 / 2, 1 / 4, 1 / 8, or 1 / 16, etc.). For example, a vector parallel to the perpendicular line of parallelogram A1B1E1C1 (normal vector) can be calculated by the cross product of vector A1B1 and vector A1C1. If the unit vector (vector with a length of 1) of the normal vector is NM and the length of vector A1E1 is |A1E1|, then vector E1E1j is determined by k × |A1E1| × NM (e.g., k = ±n / 2, ±n / 4, ±n / 8, ±n / 16, ... (n = 1, 2, 3, ...)). Also, the vector E1E1j may be set to k times the vector NM (for example, k=±16, ±32, ±48, ±64, . . . ) without using the line segment A1E1 or the line segment B1C1.

[0292] In addition, just as at least one predicted value is determined using the prediction method shown in Figure 33 in Figure 34, the encoding device or decoding device may determine at least one predicted value Ei and Eij (e.g., i = 1, 2, 3, j = 1, 2, 3, 4) using the prediction method shown in Figure 37, and use the average value of the predicted values ​​Ei, Ei1, Ei2, Ei3 and Ei4 as a candidate predicted value.

[0293] Alternatively, the encoding device or decoding device may assign a unique index value to each predicted value, notify the decoding device of the index value of the prediction method used for encoding for each vertex or for each unit of multiple vertices, and switch prediction methods according to the index value during decoding to determine predicted values ​​for the coordinates of the vertices of the triangle. Note that the information notified from the encoding device to the decoding device may be any information that can identify the prediction method used for encoding, and does not necessarily have to be an index value.

[0294] In this way, by increasing the number of options for prediction values ​​that are not on the same plane as triangle A1B1C1, the encoding device or decoding device is more likely to be able to select an effective prediction method for vertex coordinates on an uneven mesh surface, potentially further reducing the amount of code required for vertex coordinates.

[0295] The vertex coordinate encoding process will be described below. The vertex coordinate encoding process can also be said to be an encoding method for encoding the coordinates of vertices included in a three-dimensional mesh.

[0296] FIG. 38 is a flowchart showing an example of processing performed by the encoding device according to this embodiment.

[0297] In step S301, the encoding device acquires connectivity information for a 3D mesh, which is information indicating the triangles that make up the 3D mesh to be encoded and how the vertices of those triangles are connected.

[0298] In step S302, the encoding device starts a loop process for each vertex to be encoded among the multiple vertices included in the three-dimensional mesh. In the loop process, the encoding device encodes the coordinates of each vertex using a prediction process. In the loop process, the encoding device focuses on each of one or more vertices to be encoded, and performs the processes of steps S303 to S306 for the focused vertices, controlling the process so that the above processes are ultimately performed for all vertices to be encoded. The vertex to be focused on is also referred to as the vertex of interest.

[0299] In step S303, the encoding device determines predicted values ​​of the coordinates of the vertex of interest.

[0300] In step S304, the encoding device encodes information about the prediction method for the vertex of interest. The information about the prediction method for the vertex of interest indicates the prediction method used when the predicted value was determined in step S303, and includes, for example, an index value indicating the prediction method for the vertex of interest.

[0301] In step S305, the encoding device calculates a prediction residual for the coordinates of the vertex of interest using the predicted value determined in step S303.

[0302] In step S306, the encoding device encodes the prediction residual of the coordinates of the vertex of interest calculated in step S305.

[0303] In step S307, the encoding device ends the loop process for each vertex. Specifically, the encoding device determines whether the processes in steps S303 to S306 have been executed for all vertices, and if not, controls the process to be executed for vertices that have not yet been executed.

[0304] The vertex coordinate decoding process will be described below. The vertex coordinate decoding process can also be said to be a decoding method for decoding the coordinates of vertices included in a three-dimensional mesh.

[0305] FIG. 39 is a flowchart showing an example of processing performed by the decoding device according to this embodiment.

[0306] In step S401, the decoding device obtains connectivity information of a three-dimensional mesh.

[0307] In step S402, the decoding device starts a loop process for each vertex to be decoded, among the multiple vertices included in the three-dimensional mesh. In the loop process, the decoding device decodes the coordinates of each vertex using a prediction process. In the loop process, the decoding device focuses on each of one or more vertices to be decoded, and performs the processes of steps S403 to S406 for the focused vertices, controlling the process so that all vertices to be decoded are ultimately processed. The vertex to be focused on is also referred to as the vertex of interest.

[0308] In step S403, the decoding device decodes information about the prediction method of the vertex of interest. The information about the prediction method of the vertex of interest includes an index value indicating the prediction method used to encode the vertex of interest.

[0309] In step S404, the decoding device determines predicted values ​​of the coordinates of the vertex of interest using the information about the prediction method decoded in step S403.

[0310] In step S405, the decoding device decodes the prediction residual of the coordinates of the vertex of interest.

[0311] In step S406, the decoding device reconstructs the vertex coordinates based on the predicted value determined in step S404 and the prediction residual decoded in step S405.

[0312] In step S407, the decoding device ends the loop process for each vertex. Specifically, the encoding device determines whether the processes in steps S403 to S406 have been executed for all vertices, and if not, controls the process to be executed focusing on the vertices that have not yet been executed.

[0313] In this way, by notifying the decoding device of the index value of the prediction method used for encoding from the encoding device, the decoding device can correctly restore the coordinates of the vertices of a triangle from a bit stream containing the coordinates of the vertices of a triangle that the encoding device encoded while switching prediction methods for purposes such as reducing the amount of code.

[0314] The processing of the encoding device will be described in detail below.

[0315] FIG. 40 is a flowchart showing an example of processing performed by the encoding device according to this embodiment.

[0316] The encoding device may execute the process shown in Fig. 40 instead of the process of step S304. Specifically, the encoding device may encode information about the prediction method only when a predetermined condition is satisfied.

[0317] As an example, the encoding device can encode vertices using either prediction method A or B. Prediction method A is, for example, the prediction method shown in Figure 34, and prediction method B is, for example, the prediction method shown in Figure 35. Furthermore, the encoding device determines that prediction method B is applicable when all of vertices A1, B1, C1, F1, and G1 shown in Figure 35 have been encoded.

[0318] The encoding device determines whether prediction method B is applicable to encoding the vertex of interest (step S311).

[0319] If it is determined that prediction method B is applicable (Yes in step S311), the encoding device encodes information regarding the prediction method determined in step S303 (i.e., information indicating whether prediction method A or prediction method B will be used for encoding (e.g., a flag or index value)) (step S312).

[0320] On the other hand, if it is determined that prediction method B is inapplicable (No in step S311), the encoding device may use prediction method A for encoding in step S303 and may not encode information regarding the prediction method (i.e., information indicating whether prediction method A or prediction method B will be used for encoding).

[0321] The process performed by the decoding device will be described in detail below.

[0322] FIG. 41 is a flowchart showing an example of processing performed by the decoding device according to this embodiment.

[0323] The decoding device may execute the process shown in FIG. 41 instead of the process in step S403.

[0324] The processing shown in FIG. 41 may be processing performed by a decoding device when decoding the coordinates of the vertices of a triangle coded by the coding method shown in FIG.

[0325] The decoding device determines whether prediction method B is applicable to decoding of the vertex of interest (step S411).

[0326] If it is determined that prediction method B is applicable (Yes in step S411), information regarding the prediction method determined by the encoding device (i.e., information indicating whether the encoding device used prediction method A or prediction method B for encoding (e.g., a flag or index value)) is decoded (step S412).

[0327] On the other hand, if it is determined that prediction method B is inapplicable (No in step S411), the encoding device may use prediction method A for encoding (i.e., the information regarding prediction indicates that "prediction method A was used for encoding") and set the information regarding prediction to prediction method A (step S413).

[0328] Note that even when vertices A1, B1, C1, and F1 have all been coded or decoded but vertex G1 has not been coded or decoded, the coding device or decoding device may use vertex B1 instead of vertex B1' and determine that prediction method B is applicable. Also, even when vertices A1, B1, C1, and G1 shown in FIG. 35 have all been coded or decoded but vertex F1 has not been coded or decoded, the coding device or decoding device may use vertex C1 instead of vertex C1' and determine that prediction method B is applicable.

[0329] Note that, as prediction method B, the prediction method shown in FIG. 35 may be applied in place of the prediction method shown in FIG. 33 in FIG. 34 to determine a plurality of predicted values ​​Ei (i=1, 2, 3, ...), and the average value of the plurality of predicted values ​​may be used as the predicted value of the coordinates of vertex D. In this case, prediction method B may be determined to be applicable when there is one or more i of vertices Ai, Bi, Ci, Fi, and Gi that have all been coded or decoded (this is condition 1). Also, when there is one or more i of vertices Gi that have not been coded or decoded (this is condition 2), vertex Bi may be used instead of vertex Bi', and prediction method B may be determined to be applicable. Also, when there are one or more i of vertices Ai, Bi, Ci, and Gi that have all been coded or decoded and vertex Fi that have not been coded or decoded (this is condition 3), vertex Ci may be used instead of vertex Ci', and prediction method B may be determined to be applicable. Alternatively, if at least one of conditions 1 to 3 is true, it may be determined that prediction method B is applicable.

[0330] Furthermore, when prediction method B is applicable, the encoding device or decoding device may use the prediction value candidates described in Fig. 36 for prediction. In this case, if the encoding device determines in step S311 (see Fig. 40) that prediction method B is applicable, the encoding device may encode in step S312 information indicating that prediction method B was used in step S303 and information (e.g., an index value) indicating which of the prediction value candidates shown in Fig. 36 was used. Furthermore, if the encoding device determines in step S311 (see Fig. 40) that prediction method B is not applicable, the encoding device may consider that prediction method A was used in step S303, and may not encode information indicating which prediction method was used.

[0331] Furthermore, if the decoding device determines in step S411 (see FIG. 41 ) that prediction method B is applicable, it may decode in step S412 information indicating that the encoding device has used prediction method B and information (e.g., an index value) indicating which of the prediction value candidates shown in FIG. 36 was used. Furthermore, if the decoding device determines in step S411 (see FIG. 41 ) that prediction method B is not applicable, it may set the information related to prediction to prediction method A in step S413.

[0332] As described above, by notifying the decoding device of the index value of the prediction method used for encoding only when multiple prediction methods are applicable, unnecessary notification of index values ​​can be avoided, potentially reducing the amount of coding.

[0333] A method for predicting the coordinates of vertices according to this embodiment will be described below.

[0334] 42 and 43 are explanatory diagrams showing an example of a method for predicting vertex coordinates according to this embodiment. Figures 42 and 43 show an example of a method for predicting the coordinates of triangle vertices in the encoding or decoding process of a three-dimensional mesh made up of multiple triangles. In this example, the coordinates of the vertices to be encoded or decoded are predicted using connection information indicating how the triangles or their vertices are connected.

[0335] In FIG. 42 , vertices A1, B1, and C2 are vertices whose coordinates have already been coded or decoded. Vertex D is the vertex whose coordinates will be coded or decoded next. The coordinates of vertex E1 are predicted values ​​of the coordinates of vertex D, and may be derived, for example, as the coordinate values ​​of D when it is assumed that quadrilaterals C1A1B1D and C2C1B1D are parallelograms. In this case, the coordinates of vertex E1 are derived by adding vector B1E1, which has half the magnitude of vector A1C2, to the coordinates of vertex B1. In other words, the coordinates of vertex E1 are derived by adding half the difference between the coordinates of vertex C2 and vertex A1 to the coordinates of vertex B1 (see (Equation 1)).

[0336] E1 = B1 + (C2 - A1) / 2 (Formula 1)

[0337] In other words, the coordinates of vertex E1 are calculated so that the difference between the coordinates of vertex C2 and vertex A1 is twice the difference between the coordinates of vertex E1 and vertex B1.

[0338] The encoding device or decoding device may also encode or decode the vector E1D as a prediction residual.

[0339] The prediction shown in Figure 42 can be applied if vertices A1, B1, and C2 are already coded or decoded vertices, and a predicted value E1 is determined thereby. Since the start point A1 and end point C2 of vector A1C2 are directly connected to the start point B1 and end point D of vector B1D, it is considered that the correlation between the direction and magnitude of vector A1C2 and vector B1D is high. Therefore, by having an encoding device or decoding device determine vertex E1 based on vector A1C2 and vertex B1, it is highly likely that the coordinates of vertex D can be predicted with high accuracy. Furthermore, since the prediction method shown in Figure 42 can be applied under different conditions than the prediction methods shown in Figures 33, 34, or 35, using it in combination with at least one of the prediction methods shown in Figures 33, 34, or 35 can improve the frequency or possibility of determining the predicted value E1, and potentially reduce the amount of code required to encode vector E1D as a prediction residual.

[0340] In FIG. 43 , vertices A2, C2, and B1 are vertices whose coordinates have already been coded or decoded. Vertex D is the vertex whose coordinates will be coded or decoded next. The coordinates of vertex E2 are predicted values ​​of the coordinates of vertex D, and may be derived, for example, as the coordinate values ​​of D when it is assumed that quadrilaterals B2A2C2D and B1B2C2D are parallelograms. In this case, the coordinates of vertex E2 are derived by adding vector C2E2, which has half the magnitude of vector A2B1, to the coordinates of vertex C2. In other words, the coordinates of vertex E2 are derived by adding half the difference between the coordinates of vertex B1 and vertex A2 to the coordinates of vertex C2 (see (Equation 2)).

[0341] E2 = C2 + (B1 - A2) / 2 (Equation 2)

[0342] In other words, the coordinates of vertex E2 are calculated so that the difference between the coordinates of vertex B1 and vertex A2 is twice the difference between the coordinates of vertex E2 and vertex C2.

[0343] Furthermore, the encoding device or decoding device may encode or decode the vector E2D as a prediction residual.

[0344] The prediction shown in Figure 43 can be applied if vertices A2, C2, and B1 are already coded or decoded vertices, and a predicted value E2 is determined thereby. Since the start point A2 and the end point B1 of vector A2B1 are directly connected to the start point C2 and the end point D of vector C2D, it is considered that the correlation between the direction and magnitude of vector A2B1 and vector C2D is high. Therefore, by determining vertex E2 based on vector A2B1 and vertex C2, an encoding device or a decoding device is likely to be able to accurately predict the coordinates of vertex D. Furthermore, since the prediction method shown in Figure 43 can be applied under different conditions than the prediction methods shown in Figures 33, 34, or 35, using it in combination with at least one of the prediction methods shown in Figures 33, 34, or 35 may improve the frequency or possibility of determining the predicted value E2, and potentially reduce the amount of code required to encode vector E2D as a prediction residual.

[0345] 34, multiple predicted values ​​Ei may be determined using the prediction method shown in FIG. 35, and the average value of the multiple predicted values ​​may be used as the predicted value for the coordinate of vertex D. The number of predicted values ​​may be set according to the number of triangles AiBiCi to which the prediction method shown in FIG. 42 or 43 can be applied. The number of predicted values ​​may be 1. Furthermore, it is not necessary to use all triangles AiBiCi to which the prediction method shown in FIG. 42 or 43 can be applied, and a predetermined number or less of triangles AiBiCi may be used for prediction.

[0346] The encoding device or decoding device may predict the coordinates of the vertices of a triangle by fixedly applying one of the prediction methods described above, or may assign a unique index value to each method, and notify the decoding device of the index value of the prediction method used for encoding for each vertex or for each group of vertices, etc., and switch prediction methods according to the index value during decoding to determine the predicted values ​​of the coordinates of the vertices of the triangle. By notifying the decoding device of the index value of the prediction method used for encoding, it is possible to select a more effective prediction method while encoding the 3D mesh, potentially reducing the amount of code. Note that the information notified from the encoding device to the decoding device may be any information that can identify the prediction method used for encoding, and does not necessarily have to be an index value. Furthermore, when determining whether vertices Ai, Bi, Ci, Fi, and Gi have been encoded or decoded, if there is no vertex corresponding to that position, such as if there is a hole in the 3D mesh, it may be determined that the vertices have not been encoded / decoded.

[0347] The process of determining the predicted value will be described below.

[0348] FIG. 44 is a flowchart showing an example of a process for determining a predicted value according to this embodiment.

[0349] The encoding device can replace the processes of steps S303 to S304 in the encoding process shown in Figure 38 with the process shown in Figure 44. Furthermore, the decoding device can replace the processes of steps S403 to S404 in the decoding process shown in Figure 39 with the process shown in Figure 44. By performing the process shown in Figure 44, the encoding device or decoding device can encode or decode the coordinates of the vertices of a triangle while switching the method of determining predicted values ​​depending on the number of triangles AiBiCi to which each of the multiple prediction methods can be applied.

[0350] For example, when the prediction method shown in Figure 33 is set to prediction method A and all of the vertices Ai, Bi, and Ci (i = 1, 2, 3, ...) shown in Figure 35 have been coded or decoded, the encoding device or decoding device determines that prediction method A is applicable to triangle AiBiCi. Also, when the prediction method shown in Figure 42 is set to prediction method B and all of the vertices Ai, Bi, and Cj (j = i + 1, i = 1, 2, 3, ...) shown in Figure 35 have been coded or decoded, the encoding device or decoding device determines that prediction method B is applicable to triangle AiBiCi.

[0351] As shown in FIG. 44, in step S501, the encoding device or decoding device counts the number A of triangles AiBiCi to which prediction method A can be applied.

[0352] In step S502, the encoding device or decoding device counts the number B of triangles AiBiCi to which prediction method B can be applied.

[0353] In step S503, the encoding device or decoding device determines whether the number A is greater than 0 (i.e., whether there is at least one triangle AiBiCi to which prediction method A can be applied, or whether there is none). If it is determined that the number A is greater than 0 (Yes in step S503), the process proceeds to step S504; if not (No in step S503), the process proceeds to step S511.

[0354] In step S504, the encoding device or decoding device determines whether or not the number A is greater than the number B. If it is determined that the number A is greater than the number B (Yes in step S504), the process proceeds to step S505; otherwise (No in step S504), the process proceeds to step S512.

[0355] In step S505, the encoding device or decoding device determines a predicted value using prediction method A.

[0356] In step S511, the encoding device or decoding device determines whether the number B is greater than 0 (i.e., whether there is at least one triangle AiBiCi to which prediction method B can be applied, or whether there is none). If it is determined that the number B is greater than 0 (Yes in step S511), the process proceeds to step S512; if not (No in step S511), the process proceeds to step S521.

[0357] In step S512, the encoding device or decoding device determines a predicted value using prediction method B.

[0358] In step S521, the encoding device or decoding device determines a predicted value using a predetermined method. The predetermined method may be a method in which the coordinates of a vertex that is located near vertex Bi or vertex Ci and has been coded or decoded are used as the predicted value. Alternatively, the predetermined method may be a method in which, for example, when there is no coded or decoded vertex near vertex Bi or vertex Ci, the predicted value is set to predetermined coordinates such as the origin (0,0,0).

[0359] By following the above procedure, the encoding device or decoding device can switch between prediction methods depending on the number of triangles AiBiCi to which each of the multiple prediction methods can be applied, thereby selecting a prediction method that is expected to be highly effective without notifying the index value of the prediction method, etc., and potentially reducing the amount of code required to encode the prediction residual.

[0360] In this example, the same effect may be obtained by using prediction method B instead of the prediction method shown in Figure 42, and determining that prediction method B is applicable to triangle AiBiCi when all of the vertices Ai, Bj, Ci (j = i - 1, i = 2, 3, ...) shown in Figure 35 have been encoded or decoded.

[0361] It should be noted that the determination condition in step S504 can also be set to "number A≧number B," that is, "whether number A is equal to or greater than number B." However, in cases where prediction method B is expected to have higher prediction accuracy than prediction method A, such as when the prediction method shown in Fig. 33 is used as prediction method A and the prediction method shown in Fig. 42 or 43 is used as prediction method B, determining a predicted value using prediction method B when numbers A and B are the same may be able to further reduce the amount of code required to encode the prediction residual.

[0362] The encoding device or the decoding device can also be described as follows.

[0363] (1) An encoding device comprising: a circuit; and a memory connected to the circuit, wherein, in an operation of encoding coordinates of vertices of polygons forming a surface of a three-dimensional mesh, the circuit selects one from two or more prediction processes, including a first prediction process that determines, based on coordinates of at least one second vertex of a second polygon adjacent to a first polygon including a first vertex to be encoded, at least one coordinate on a plane including the second polygon as a first predicted value of the coordinate of the first vertex, and a second prediction process that determines, based on coordinates of at least one coordinate not on the plane including the second polygon, as a second predicted value of the coordinate of the first vertex, and encodes selection information indicating the selected prediction process and a difference value between the coordinate of the first vertex and a third predicted value determined by the selected prediction process.

[0364] (2) In the encoding device described in (1), the circuit determines the second predicted value in the second prediction process based on the coordinates of at least one second vertex of at least one third polygon other than the first polygon that is adjacent to the second polygon and does not contact the second polygon.

[0365] (3) In the encoding device described in (2), when at least one of the second vertices has already been encoded, the circuit selects one of the two or more prediction processes and encodes the selection information, and when none of the second vertices has already been encoded, it selects the first prediction process and does not encode the selection information.

[0366] (4) In the encoding device described in (2) or (3), the circuit includes a third prediction process that determines the coordinates of a straight line passing through the first predicted value and the second predicted value as a fourth predicted value of the coordinates of the first vertex.

[0367] (5) In the encoding device described in (1), in the second prediction process, the circuit determines at least one coordinate on a line that intersects a plane containing the second polygon perpendicularly with the coordinate of the first predicted value or near the coordinate of the first predicted value as the second predicted value.

[0368] (6) A decoding device comprising: a circuit; and a memory connected to the circuit, wherein the circuit, in an operation of restoring coordinates of vertices of polygons forming the surface of a three-dimensional mesh, comprises two or more prediction processes, including a first prediction process that determines at least one coordinate on a plane including a second polygon as a first predicted value of the coordinate of the first vertex based on coordinates of at least one second vertex of a second polygon adjacent to the first polygon including the first vertex to be restored, and a second prediction process that determines at least one coordinate not on the plane including the second polygon as a second predicted value of the coordinate of the first vertex; the decoding device decodes selection information and a difference value indicating one of the two or more prediction processes, determines a third predicted value using the prediction process indicated by the selection information, and restores the coordinate of the first vertex based on the third predicted value and the difference value.

[0369] (7) A decoding device according to (6), wherein the circuit determines the second predicted value in the second prediction process based on the coordinates of at least one second vertex of at least one third polygon other than the first polygon adjacent to the second polygon, the second vertex not touching the second polygon.

[0370] (8) In the decoding device described in (7), when at least one of the second vertices has been restored, the circuit decodes the selection information and determines a third predicted value by the indicated prediction process, and when none of the second vertices has been encoded, the circuit determines a third predicted value by the first prediction process without decoding the selection information.

[0371] (9) The encoding device described in (7) or (8), wherein the circuit includes a third prediction process that determines the coordinates of a straight line passing through the first predicted value and the second predicted value as a fourth predicted value of the coordinates of the first vertex.

[0372] (10) A decoding device according to (6), wherein the circuit determines, in the second prediction process, at least one coordinate on a line that intersects a plane containing the second polygon perpendicularly with the coordinate of the first predicted value or in the vicinity thereof, as the second predicted value.

[0373] The encoding device or the decoding device can also be described as follows.

[0374] (1) An encoding device comprising a circuit and a memory connected to the circuit, wherein, in an operation of encoding coordinates of vertices of polygons forming the surface of a three-dimensional mesh, the circuit determines at least one coordinate on a plane including at least one second vertex of a second polygon adjacent to a first polygon including a first vertex to be encoded and not tangent to the first polygon, at least one third vertex of a third polygon different from the second polygon adjacent to the first polygon and not tangent to the first polygon, and at least one fourth vertex of a vertex of the first polygon and not tangent to the third polygon as a predicted value of the coordinate of the first vertex, and encodes a difference value between the coordinate of the first vertex and the predicted value.

[0375] (2) In the encoding device described in (1), the circuit determines at least one coordinate on a first line that is parallel to a first vector that starts at the second vertex and ends at the third vertex and passes through the fourth vertex as a predicted value of the coordinate of the first vertex.

[0376] (3) In the encoding device described in (2), the circuit determines the coordinate that is a distance of 1 / 2 the length of the first vector from the fourth vertex in the direction of the first vector as the predicted value of the coordinate of the first vertex.

[0377] (4) A decoding device comprising a circuit and a memory connected to the circuit, wherein the circuit, in an operation of restoring coordinates of vertices of polygons forming the surface of a three-dimensional mesh, determines at least one coordinate on a plane including at least one second vertex of a second polygon adjacent to a first polygon including a first vertex to be restored, the second vertex not tangent to the first polygon, at least one third vertex of a third polygon different from the second polygon adjacent to the first polygon, the third vertex not tangent to the first polygon, and at least one fourth vertex of a vertex of the first polygon not tangent to the third polygon as a predicted value of the coordinate of the first vertex; decodes a difference value corresponding to the coordinate of the first vertex; and restores the coordinate of the first vertex based on the predicted value and the difference value.

[0378] (5) A decoding device according to (4), wherein the circuit determines at least one coordinate on a first line that is parallel to a first vector that starts at the second vertex and ends at the third vertex and passes through the fourth vertex as a predicted value of the coordinate of the first vertex.

[0379] (6) In the decoding device described in (5), the circuit determines a coordinate that is a distance of 1 / 2 the length of the first vector from the fourth vertex in the direction of the first vector as a predicted value of the coordinate of the first vertex.

[0380] Another example of the encoding process according to this embodiment will be described below.

[0381] FIG. 45 is a flowchart showing an example of encoding processing according to this embodiment.

[0382] The encoding process shown in Fig. 45 can also be said to be an encoding method for encoding the coordinates of vertices included in a three-dimensional mesh. The encoding process shown in Fig. 45 is executed by an encoding device.

[0383] In step S601, the encoding device derives a predicted value of the coordinates of the first vertex using the coordinates of three vertices. The three vertices are three vertices (e.g., vertices A1, B1, and C2 in FIG. 42) that are different from the first vertex (e.g., vertex D in FIG. 42) among multiple vertices connected to a second vertex (e.g., vertex C1 in FIG. 42) that is connected to the first vertex to be encoded.

[0384] In step S602, the encoding device encodes the difference between the predicted value derived in step S601 and the coordinate of the first vertex.

[0385] As a result, the encoding device derives a predicted value of the coordinates of the first vertex using the coordinates of three vertices other than the first vertex among multiple vertices connected to a second vertex connected to the first vertex to be encoded. When the coordinates of the three vertices are relatively closely related to the position of the first vertex, the derived predicted value may be an appropriate value that is relatively close to the coordinates of the three vertices. This may improve the accuracy of encoding the first vertex and may also reduce the amount of code. In this way, the encoding device may contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh.

[0386] For example, the encoding device may derive the predicted value without using the coordinates of the second vertex.

[0387] As a result, the encoding device may be able to more easily derive an appropriate predicted value by using the coordinates of the three vertices when deriving a predicted value, in other words, by not using the "second vertex connected to the first vertex to be encoded." In this way, the encoding device may more easily contribute to improving the encoding process for the coordinates of the vertices of a 3D mesh.

[0388] For example, when deriving a predicted value, the encoding device may use three vertices, including at least a third vertex (e.g., vertex B1 in Figure 42) and a fourth vertex (e.g., vertex C2 in Figure 42), which are two vertices connected to the first vertex, to derive the predicted value.

[0389] As a result, when deriving a predicted value, the encoding device may be able to more easily derive an appropriate predicted value by using the coordinates of at least three vertices, including the third vertex and the fourth vertex, which are the two vertices connected to the first vertex. In this way, the encoding device may more easily contribute to improving the encoding process for the coordinates of the vertices of a 3D mesh.

[0390] For example, when deriving a predicted value, the encoding device may derive the predicted value using three vertices that include at least a fifth vertex (e.g., vertex A1 in Figure 42), which is a vertex that is not connected to the first vertex.

[0391] As a result, the encoding device may be able to more easily derive an appropriate predicted value by using the coordinates of at least three vertices, including the fifth vertex that is not connected to the first vertex, when deriving a predicted value. In this way, the encoding device may more easily contribute to improving the encoding process for the coordinates of the vertices of a 3D mesh.

[0392] For example, when deriving a predicted value, the encoding device derives the predicted value using three vertices, including a fifth vertex (e.g., vertex A1 in Figure 42) that is not connected to the first vertex, and the difference value between the coordinates of the fourth vertex and the coordinates of the fifth vertex may be twice the difference value between the predicted value of the coordinates of the first vertex and the coordinates of the third vertex.

[0393] As a result, when deriving a predicted value, the encoding device may be able to more easily derive an appropriate predicted value by calculating the coordinates of one of the four vertices of a trapezoid formed by three vertices, including the third vertex and the fourth vertex, which are two vertices connected to the first vertex, and the fifth vertex, which is a vertex not connected to the first vertex. In this way, the encoding device may more easily contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh.

[0394] For example, the predicted value may be a first predicted value. The encoding device may further derive second predicted values ​​that are predicted values ​​of the coordinates of the first vertex (e.g., vertex D in FIG. 43 ) using coordinates of three new vertices (e.g., vertices A2, B1, and C2 in FIG. 43 ) that are different from the first vertex, among a plurality of new vertices that are connected to a new second vertex (e.g., vertex C1 in FIG. 43 ) that is connected to the first vertex (e.g., vertex D in FIG. 43 ) and that are at least partially different from the plurality of new vertices, derive third predicted values ​​that are new predicted values ​​of the coordinates of the first vertex using the first predicted value and the second predicted value, and encode a new difference value between the third predicted value and the coordinates of the first vertex.

[0395] As a result, the encoding device derives a new third predicted value using a second predicted value derived using three new vertices that are at least partially different from the three vertices. The derived third predicted value may be a more appropriate predicted value than the first predicted value or the second predicted value. This may improve the accuracy of encoding the first vertex and may reduce the amount of code. In this way, the encoding device may contribute to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh.

[0396] For example, when deriving the third predicted value, the encoding device may derive the average of the first predicted value and the second predicted value as the third predicted value.

[0397] This allows the encoding device to more easily calculate the third predicted value by performing information processing to take the average of the first predicted value and the second predicted value, thereby contributing to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh through easier information processing.

[0398] For example, when deriving the third predicted value, the encoding device may derive a weighted average of the first predicted value and the second predicted value as the third predicted value.

[0399] This allows the encoding device to more easily calculate the third predicted value by performing information processing to take a weighted average of the first predicted value and the second predicted value, thereby contributing to improving the encoding process for the coordinates of the vertices of a three-dimensional mesh through easier information processing.

[0400] For example, the encoding device may further acquire connection information indicating the connections between vertices included in the three-dimensional mesh, and when deriving a predicted value, identify the three vertices using the connections between the vertices included in the three-dimensional mesh indicated in the acquired connection information, and derive a predicted value for the coordinates of the first vertex using the coordinates of the identified three vertices.

[0401] This allows the encoding device to more easily identify the three vertices by using the connection information and derive a predicted value using the identified three vertices, which can contribute to improving the encoding process for the coordinates of the vertices of a 3D mesh more easily.

[0402] Another example of the decoding process according to this embodiment will be described below.

[0403] FIG. 46 is a flowchart showing an example of the decoding process according to this embodiment.

[0404] The decoding process shown in Fig. 46 can also be said to be a decoding method for decoding the coordinates of vertices included in a three-dimensional mesh. The decoding process shown in Fig. 46 is executed by a decoding device.

[0405] In step S701, the decoding device derives a predicted value of the coordinates of the first vertex using the coordinates of three vertices. The three vertices are three vertices (e.g., vertices A1, B1, and C2 in FIG. 42 ) different from the first vertex (e.g., vertex D in FIG. 42 ) among multiple vertices connected to a second vertex (e.g., vertex C1 in FIG. 42 ) connected to the first vertex to be decoded.

[0406] In step S702, the decoding device decodes the difference value between the predicted value and the coordinate of the first vertex.

[0407] In step S703, the decoding device reconstructs the coordinates of the first vertex using the predicted value and the difference value.

[0408] As a result, the decoding device derives a predicted value of the coordinates of the first vertex using the coordinates of three vertices different from the first vertex among multiple vertices connected to a second vertex connected to the first vertex to be decoded. When the correlation between the coordinates of the three vertices and the position of the first vertex is relatively strong, the derived predicted value may be an appropriate value that is relatively close to the coordinates of the three vertices. This may improve the accuracy of decoding the first vertex and may also reduce the amount of code. In this way, the decoding device may contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh.

[0409] For example, the decoding device may derive the predicted value without using the coordinates of the second vertex.

[0410] As a result, the decoding device may be able to more easily derive an appropriate predicted value by using the coordinates of the three vertices when deriving a predicted value, in other words, by not using the "second vertex connected to the first vertex to be decoded." In this way, the decoding device may more easily contribute to improving the decoding process for the coordinates of the vertices of a 3D mesh.

[0411] For example, when deriving a predicted value, the decoding device may use three vertices, including at least a third vertex (e.g., vertex B1 in Figure 42) and a fourth vertex (e.g., vertex C2 in Figure 42), which are two vertices connected to the first vertex, to derive the predicted value.

[0412] As a result, when deriving a predicted value, the decoding device may be able to more easily derive an appropriate predicted value by using the coordinates of at least three vertices, including the third vertex and the fourth vertex, which are two vertices connected to the first vertex. In this way, the decoding device may more easily contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh.

[0413] For example, when deriving a predicted value, the decoding device may use three vertices that include at least a fifth vertex (e.g., vertex A1 in Figure 42), which is a vertex that is not connected to the first vertex, as the three vertices.

[0414] As a result, the decoding device may be able to more easily derive an appropriate predicted value by using the coordinates of at least three vertices, including the fifth vertex that is not connected to the first vertex, when deriving a predicted value. In this way, the decoding device may more easily contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh.

[0415] For example, when deriving a predicted value, the decoding device may derive the predicted value using three vertices that further include a fifth vertex (e.g., vertex A1 in Figure 42) that is not connected to the first vertex, and the difference value between the coordinates of the fourth vertex and the coordinates of the fifth vertex may be twice the difference value between the predicted value of the coordinates of the first vertex and the coordinates of the third vertex.

[0416] As a result, when deriving a predicted value, the decoding device may be able to more easily derive an appropriate predicted value by calculating the coordinates of one of the four vertices of a trapezoid formed by three vertices, including the third vertex and the fourth vertex, which are two vertices connected to the first vertex, and the fifth vertex, which is a vertex not connected to the first vertex.In this way, the decoding device can more easily contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh.

[0417] For example, the predicted value may be a first predicted value. The decoding device may further derive a second predicted value that is a predicted value of the coordinates of the first vertex (e.g., vertex A2, B1, and C2 in FIG. 43 ) using coordinates of three new vertices (e.g., vertices A2, B1, and C2 in FIG. 43 ) that are different from the first vertex, among a plurality of new vertices that are connected to a new second vertex (e.g., vertex C1 in FIG. 43 ) that is connected to the first vertex (e.g., vertex D in FIG. 43 ) and that are at least partially different from the plurality of new vertices, derive a third predicted value that is a new predicted value of the coordinates of the first vertex using the first predicted value and the second predicted value, decode a new difference value between the third predicted value and the coordinate of the first vertex, and restore the coordinate of the first vertex using the third predicted value and the new difference value.

[0418] As a result, the decoding device derives a new third predicted value using a second predicted value derived using three new vertices that are at least partially different from the above three vertices. The derived third predicted value may be a more appropriate predicted value than the first predicted value or the second predicted value. This may improve the accuracy of decoding the first vertex and may reduce the amount of code. In this way, the decoding device may contribute to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh.

[0419] For example, when deriving the third predicted value, the decoding device may derive the average of the first predicted value and the second predicted value as the third predicted value.

[0420] This allows the decoding device to more easily calculate the third predicted value by performing information processing to average the first predicted value and the second predicted value, thereby contributing to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh through easier information processing.

[0421] For example, when deriving the third predicted value, the decoding device may derive a weighted average of the first predicted value and the second predicted value as the third predicted value.

[0422] This allows the decoding device to more easily calculate the third predicted value by performing information processing to take a weighted average of the first predicted value and the second predicted value, thereby contributing to improving the decoding process for the coordinates of the vertices of a three-dimensional mesh through easier information processing.

[0423] For example, the decoding device may further acquire connection information indicating the connections of the vertices included in the three-dimensional mesh, and when deriving a predicted value, identify three vertices using the connections between the vertices included in the three-dimensional mesh indicated in the acquired connection information, and derive a predicted value of the coordinates of the first vertex using the coordinates of the identified three vertices.

[0424] According to the above aspect, the decoding device can more easily identify three vertices by using the connection information and derive a predicted value using the identified three vertices, which can contribute to improving the decoding process for the coordinates of the vertices of a 3D mesh more easily.

[0425] <Other Examples> Although the aspects of the encoding device and the decoding device have been described above according to the embodiments, the aspects of the encoding device and the decoding device are not limited to the embodiments. Modifications that a person skilled in the art can conceive of may be applied to the embodiments, and multiple components in the embodiments may be combined in any manner.

[0426] For example, a process performed by a specific component in the embodiment may be performed by another component instead of the specific component. Also, the order of multiple processes may be changed, or multiple processes may be performed in parallel.

[0427] Furthermore, as described above, at least some of the configurations of the present disclosure may be implemented as an integrated circuit. At least some of the processes of the present disclosure may be used as an encoding method or a decoding method. A program for causing a computer to execute the encoding method or the decoding method may be used. A non-transitory computer-readable recording medium on which the program is recorded may be used. A bitstream for causing a decoding device to perform a decoding process may be used.

[0428] Furthermore, at least some of the configurations and processes of the present disclosure may be used as a transmitting device, a receiving device, a transmitting method, or a receiving method. A program for causing a computer to execute the transmitting method or the receiving method may be used. Furthermore, a non-transitory computer-readable recording medium on which the program is recorded may be used.

[0429] In the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that realizes the encoding device and the like in the above embodiments is the following program.

[0430] In other words, the software is a program that causes a computer to execute an encoding method for encoding the coordinates of vertices included in a three-dimensional mesh, in which a predicted value of the coordinates of the first vertex is derived using the coordinates of three vertices different from the first vertex out of multiple vertices connected to a second vertex that is connected to a first vertex to be encoded, and the differential value between the predicted value and the coordinate of the first vertex is encoded.

[0431] The software for realizing the decoding device and the like according to the above-described embodiment is the following program.

[0432] In other words, the software is a program that causes a computer to execute a decoding method for decoding the coordinates of vertices included in a three-dimensional mesh, in which a predicted value of the coordinates of the first vertex is derived using the coordinates of three vertices different from the first vertex out of multiple vertices connected to a second vertex connected to the first vertex to be decoded, the difference value between the predicted value and the coordinate of the first vertex is decoded, and the coordinate of the first vertex is restored using the predicted value and the difference value.

[0433] The present disclosure is useful, for example, in encoding devices, decoding devices, transmitting devices, receiving devices, etc. related to three-dimensional meshes, and is applicable to computer graphics systems, three-dimensional data display systems, etc.

[0434] 100 Encoding device 101, 121, 144 Vertex information encoder 102, 145 Connection information encoder 103, 122 Attribute information encoder 104, 204, 1102 Preprocessor 105, 205, 2107 Postprocessor 110 Three-dimensional data encoding system 111, 211 Controller 112, 212 Input / output processor 113 Three-dimensional data encoder 114 System multiplexer 115 Three-dimensional data generator 123 Metadata encoder 124 Multiplexer 131 Vertex image generator 132 Attribute image generator 133 Metadata generator 134 Video encoder 141 Two-dimensional data encoder 142 Mesh data encoder 143 Texture encoder 148 Description encoder 151, 251 Circuit 152, 252 Memory 200 Decoding device 201, 221, 244 Vertex information decoder 202, 245 Connection information decoder 203, 222 Attribute information decoder 210 3D data decoding system 213 3D data decoder 214 System demultiplexer 215, 247 Presentation device 216 User interface 223 Metadata decoder 224 Demultiplexer 231 Vertex information generator 232 Attribute information generator 234 Video decoder 241 2D data decoder 242 Mesh data decoder 243 Texture decoder 246 Mesh reconstructor 248 Description decoder 300 Network 310 External connector 1101 Input mesh 1103, 2103 Base mesh 1104, 2104 Displacement data 1105, 2105 Texture data 1106, 2106 Attribute map 1107 Compressor 1108, 2101, 1304 Bitstream 1301, 2302, 2304 Mesh frame 1302, 2301 Base mesh frame 1303, 2303 Displacement information 2102 Decompressor 2108 Output mesh

Claims

1. A coding method for encoding the coordinates of vertices included in a three-dimensional mesh, the method comprising: deriving a predicted value of the coordinates of the first vertex using the coordinates of three vertices different from the first vertex out of multiple vertices connected to a second vertex connected to a first vertex to be encoded; and encoding the difference between the predicted value and the coordinates of the first vertex.

2. The encoding method according to claim 1, wherein the predicted value is derived without using the coordinates of the second vertex.

3. The encoding method according to claim 1, wherein when deriving the predicted value, the predicted value is derived using three vertices including at least a third vertex and a fourth vertex, which are two vertices connected to the first vertex.

4. The encoding method according to claim 1, wherein when deriving the predicted value, the predicted value is derived using three vertices including at least a fifth vertex that is not connected to the first vertex.

5. The encoding method according to claim 3, wherein when deriving the predicted value, the predicted value is derived using three vertices including a fifth vertex that is not connected to the first vertex, and the difference between the coordinates of the fourth vertex and the coordinates of the fifth vertex is twice the difference between the predicted value of the coordinates of the first vertex and the coordinates of the third vertex.

6. The encoding method according to claim 1, wherein the predicted value is a first predicted value, and the encoding method further comprises: deriving a second predicted value that is a predicted value of the coordinates of the first vertex using coordinates of three new vertices different from the first vertex among a plurality of new vertices that are connected to a new second vertex that is connected to the first vertex and that are at least partially different from the first vertex; deriving a third predicted value that is a new predicted value of the coordinates of the first vertex using the first predicted value and the second predicted value; and encoding a new difference value between the third predicted value and the coordinates of the first vertex.

7. The encoding method according to claim 6, wherein when deriving the third predicted value, an average of the first predicted value and the second predicted value is derived as the third predicted value.

8. The encoding method according to claim 6, wherein when deriving the third predicted value, a weighted average of the first predicted value and the second predicted value is derived as the third predicted value.

9. The encoding method according to claim 1, further comprising: acquiring connection information indicating the connections of the vertices included in the three-dimensional mesh; and, when deriving the predicted value, identifying the three vertices using the connections between the vertices included in the three-dimensional mesh indicated in the acquired connection information; and deriving the predicted value of the coordinates of the first vertex using the coordinates of the identified three vertices.

10. A decoding method for decoding the coordinates of vertices included in a three-dimensional mesh, comprising: deriving a predicted value of the coordinates of the first vertex using the coordinates of three vertices different from the first vertex among a plurality of vertices connected to a second vertex connected to a first vertex to be decoded; decoding a difference value between the predicted value and the coordinate of the first vertex; and restoring the coordinate of the first vertex using the predicted value and the difference value.

11. The decoding method according to claim 10, wherein the predicted value is derived without using the coordinates of the second vertex.

12. The decoding method according to claim 10, wherein when deriving the predicted value, the predicted value is derived using three vertices including at least a third vertex and a fourth vertex, which are two vertices connected to the first vertex.

13. The decoding method according to claim 10, wherein when deriving the predicted value, the predicted value is derived using three vertices including at least a fifth vertex that is not connected to the first vertex.

14. The decoding method of claim 12, wherein when deriving the predicted value, the predicted value is derived using three vertices that further include a fifth vertex that is not connected to the first vertex, and the difference value between the coordinates of the fourth vertex and the coordinates of the fifth vertex is twice the difference value between the predicted value of the coordinates of the first vertex and the coordinates of the third vertex.

15. The decoding method according to claim 10, wherein the predicted value is a first predicted value, and the decoding method further includes deriving a second predicted value that is a predicted value of the coordinates of the first vertex using coordinates of three new vertices different from the first vertex among a plurality of new vertices that are connected to a new second vertex that is connected to the first vertex and that are at least partially different from the first vertex; deriving a third predicted value that is a new predicted value of the coordinates of the first vertex using the first predicted value and the second predicted value; decoding a new difference value between the third predicted value and the coordinate of the first vertex; and restoring the coordinate of the first vertex using the third predicted value and the new difference value.

16. The decoding method according to claim 15, wherein when deriving the third predicted value, an average of the first predicted value and the second predicted value is derived as the third predicted value.

17. The decoding method according to claim 15, wherein when deriving the third predicted value, a weighted average of the first predicted value and the second predicted value is derived as the third predicted value.

18. The decoding method according to claim 10, further comprising: acquiring connection information indicating the connections of the vertices included in the three-dimensional mesh; and, when deriving the predicted value, identifying the three vertices using the connections between the vertices included in the three-dimensional mesh indicated in the acquired connection information; and deriving the predicted value of the coordinates of the first vertex using the coordinates of the identified three vertices.

19. An encoding device comprising: a memory; and a circuit capable of accessing the memory, wherein the circuit executes an encoding method in operation for encoding coordinates of vertices included in a three-dimensional mesh, the encoding method deriving a predicted value of the coordinates of the first vertex using coordinates of three vertices different from the first vertex out of a plurality of vertices connected to a second vertex connected to a first vertex to be encoded; and encoding the difference value between the predicted value and the coordinate of the first vertex.

20. A decoding device comprising: a memory; and a circuit capable of accessing the memory, wherein the circuit executes a decoding method in operation for decoding coordinates of vertices included in a three-dimensional mesh, the decoding method deriving a predicted value of the coordinates of the first vertex using coordinates of three vertices different from the first vertex out of a plurality of vertices connected to a second vertex connected to a first vertex to be decoded; decoding a difference value between the predicted value and the coordinate of the first vertex; and restoring the coordinate of the first vertex using the predicted value and the difference value.